[unraid] rclone Permission and writefilehandle.write Issue

Hello, I am replicating the issue in the Linux Environment (without Samba). Could you please tell me the "please share the full debug log", How to do so?

I am using -vv though.

Thanks.

Use -vv and use --log-file and share the log via pastebin/drivelink/however you want to share it.

1 Like

Okay, @Animosity022 I have added the --vfs-cache-mode writes into my command.

Command Now: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --fast-list --uid 99 --gid 100 --umask 002 -vv --vfs-cache-mode writes --log-file /mnt/disks/ua_hdd1/rclone.log gdrive: /mnt/disks/ua_hdd1/gdrive &

Steps/Process Below:

I have added the 10GB.bin file for testing here: /mnt/disks/ua_hdd1/

Now, I am going to copy this 10GB.bin file from the above mentioned path to the rclone mount of Google Drive i.e.: /mnt/disks/ua_hdd1/gdrive

root@unraid:~# cd /mnt/disks/ua_hdd1/
root@unraid:/mnt/disks/ua_hdd1# ls
10GB.bin  command.txt  downloads/  gdrive/  others/  rclone.log
root@unraid:/mnt/disks/ua_hdd1# cp 10GB.bin /mnt/disks/ua_hdd1/gdrive

It is copying the file to the location right now. Stops copying around 7GB

cp: error writing '/mnt/disks/ua_hdd1/gdrive/10GB.bin': Input/output error
root@unraid:/mnt/disks/ua_hdd1# 

When I do ls -la, it gives below output

root@unraid:/mnt/disks/ua_hdd1# cd gdrive
root@unraid:/mnt/disks/ua_hdd1/gdrive# ls -la
-rw-rw-r-- 1 nobody users 7512698880 Dec 27 22:18 10GB.bin

Log File: https://www.dropbox.com/s/b3eky57kvefipdh/rclone1.log?dl=0

Please check and let me know, if there is some issue, I will re-do everything. Please assist. Thanks.

You run out of space on your mount point you are copying to, so it fails:

2020/12/27 22:14:12 DEBUG : 10GB.bin(0xc0000be580): _writeAt: size=131072, off=7512653824
2020/12/27 22:14:12 DEBUG : 10GB.bin(0xc0000be580): >_writeAt: n=0, err=write /root/.cache/rclone/vfs/gdrive/10GB.bin: no space left on device
2020/12/27 22:14:12 DEBUG : &{10GB.bin (rw)}: >Write: written=0, err=write /root/.cache/rclone/vfs/gdrive/10GB.bin: no space left on device
1 Like

Okay, I have actually mounted the Google Drive on a 1TB Hard Disk, which shouldn't run out of the space.

Here is the Hard Disk:

It has more than 900GB Space left.

Please assist me with the solution. Thanks.

You need to specify a location for the cache that has enough space to store a 10GB file:

2020/12/27 22:07:08 DEBUG : vfs cache: root is "/root/.cache/rclone/vfs/gdrive"
2020/12/27 22:07:08 DEBUG : vfs cache: metadata root is "/root/.cache/rclone/vfs/gdrive"

Use --cache-dir and point it wherever you want to store cache files as your /root doesn't have 10GB of space.

1 Like

Okay, I have changed the cache dir to /mnt/disks/ua_hdd1/rclone/cache

Command Now: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --fast-list --uid 99 --gid 100 --umask 002 -vv --vfs-cache-mode writes --log-file /mnt/disks/ua_hdd1/rclone.log --cache-dir /mnt/disks/ua_hdd1/rclone/cache gdrive: /mnt/disks/ua_hdd1/gdrive &

I'm now going to do the test and see if it fails or works.....

Okay, It worked fine. No Error. Thanks for that. But, I have some more questions if you don't mind?

  1. Is there anything you suggest adding (flags/commands) to this setup to improve the performance and overall robust-ness?

  2. How can I make sure the cache director is cleaned once the work is done?

  3. Is there any other flags which makes sure, I don't have any issue because of the cache again?

  4. Anything else you wish to tell which I might have forgot to ask?

Thanks.

--fast-list does nothing and can be removed.
--buffer-size - any reason you are setting it? You should remove it.

You can turn off -vv once you are confident things are working. More is less. Use the defaults unless you have a reason to not use them.

It cleans itself and there is nothing you need to do. If you want to change any of the times, please check here:

1 Like

Okay, I have made the changes. But regarding, Buffer Size.

My Media Consumption Process:

I download my media from Sonarr/Radarr (via. Torrent/NZB) to Local HDD and then Sonarr/Radarr copies the file directly to the Mounted Gdrive (via. rclone).

Now, I use Plex to watch the Media, so I have been using Gdrive/media/shows and Gdrive/media/movies directly onto the Plex.

So, If I play a movie, it downloads from the Google's Server directly.

  1. So if you have any better flags/commands to improve this whole setup let me know?
  2. Do I need buffer-size flag for this?
  3. I have been wondering one thing for a long-time. How can I do this? (Mentioned Below)

Each time, when I play a media on the Plex, it loads from the Google's Server, right?

How can I make sure, I don't have to re-load the media for x-duration even if I turn off the Plex for the day?

Like, I am watching Tenet. I watched the movie till 1 Hour, but now I wish to watch next 1 hour tomorrow. But, Plex buffers the media for n number of minutes in advance for me, How can I make sure, I retain those n number of minutes for x duration? So that, when I re-launch/resume the Plex, it don't have to redownload?

I hope I am able to explain this.

Thanks.

Read through vfs-cache-mode full and most folks use that:

Okay, --vfs-cache-mode full is the thing I should use instead of writes. But then, How do I set the duration/age of the cache?

--vfs-cache-max-age duration

Is this something I need to use for that purpose?

Also, Where is this saving? Which location?

Thanks.

Like 3 posts up, you set the location of the cached files with --cache-dir.

Use whatever duration you'd like to keep as it's really up to you.

1 Like

Hello, I'm facing another issue. Since, I am using --vfs-cache-mode full now, I think it's trying to cache everything without even running the file?

Because of which, /root/.cache/rclone is taking up all the space, causing a lot of errors to my unraid.

Command: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --uid 99 --gid 100 --umask 002 --vfs-cache-mode full --vfs-cache-max-age 24h --cache-dir /mnt/disks/ua_hdd1/rclone/cache gdrive: /mnt/disks/ua_hdd1/gdrive &

Please Check, Because it is not even writing the cache on the given directory/folder. But instead still going onto the /root/.cache/rclone

2020/12/28 18:43:19 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:19 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo(0xc002c0d240): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:21 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:21 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo(0xc002c0db80): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:27 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:27 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo(0xc0013b0b40): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:28 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:28 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo(0xc0013b11c0): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:29 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:29 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo(0xc0013b1540): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:30 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:30 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo(0xc0013b1980): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo: no

Please check the top of your log with debug and it'll tell you where it's writing to.

1 Like

Thanks for the reply. How do I confirm if it is using the cache-dir as I have mentioned, myself and not trusting the log?

Currently, it is showing this:

root@unraid:/mnt/disks/ua_hdd1# cat rclone.log
2020/12/28 19:00:01 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "mount" "--allow-other" "--buffer-size" "256M" "--dir-cache-time" "96h" "--timeout" "1h" "--default-permissions" "--uid" "99" "--gid" "100" "--umask" "002" "--vfs-cache-mode" "full" "--vfs-cache-max-age" "24h" "--cache-dir" "/mnt/disks/ua_hdd1/rclone/cache" "-vv" "--log-file" "/mnt/disks/ua_hdd1/rclone.log" "gdrive:" "/mnt/disks/ua_hdd1/gdrive"]
2020/12/28 19:00:01 DEBUG : Creating backend with remote "gdrive:"
2020/12/28 19:00:01 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2020/12/28 19:00:02 DEBUG : vfs cache: root is "/mnt/disks/ua_hdd1/rclone/cache/vfs/gdrive"
2020/12/28 19:00:02 DEBUG : vfs cache: metadata root is "/mnt/disks/ua_hdd1/rclone/cache/vfs/gdrive"
2020/12/28 19:00:02 DEBUG : Creating backend with remote "/mnt/disks/ua_hdd1/rclone/cache/vfs/gdrive"
2020/12/28 19:00:02 DEBUG : Google drive root '': Mounting on "/mnt/disks/ua_hdd1/gdrive"
2020/12/28 19:00:02 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2020/12/28 19:00:02 DEBUG : : Root: 
2020/12/28 19:00:02 DEBUG : : >Root: node=/, err=<nil>
2020/12/28 19:00:02 DEBUG : /: Attr: 
2020/12/28 19:00:02 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Attr: 
2020/12/28 19:00:10 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: ReadDirAll: 
2020/12/28 19:00:10 DEBUG : /: >ReadDirAll: item=10, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Attr: 
2020/12/28 19:00:10 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="P"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=P/, err=<nil>
2020/12/28 19:00:10 DEBUG : P/: Attr: 
2020/12/28 19:00:10 DEBUG : P/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="Test1"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=Test1/, err=<nil>
2020/12/28 19:00:10 DEBUG : Test1/: Attr: 
2020/12/28 19:00:10 DEBUG : Test1/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="bu"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=bu/, err=<nil>
2020/12/28 19:00:10 DEBUG : bu/: Attr: 
2020/12/28 19:00:10 DEBUG : bu/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="media"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=media/, err=<nil>
2020/12/28 19:00:10 DEBUG : media/: Attr: 
2020/12/28 19:00:10 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="media_swizzin"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=media_swizzin/, err=<nil>
2020/12/28 19:00:10 DEBUG : media_swizzin/: Attr: 
2020/12/28 19:00:10 DEBUG : media_swizzin/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="nzb"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=nzb/, err=<nil>
2020/12/28 19:00:10 DEBUG : nzb/: Attr: 
2020/12/28 19:00:10 DEBUG : nzb/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="test"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=test/, err=<nil>
2020/12/28 19:00:10 DEBUG : test/: Attr: 
2020/12/28 19:00:10 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="torrent"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=torrent/, err=<nil>
2020/12/28 19:00:10 DEBUG : torrent/: Attr: 
2020/12/28 19:00:10 DEBUG : torrent/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="youtube"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=youtube/, err=<nil>
2020/12/28 19:00:10 DEBUG : youtube/: Attr: 
2020/12/28 19:00:10 DEBUG : youtube/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="yt"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=yt/, err=<nil>
2020/12/28 19:00:10 DEBUG : yt/: Attr: 
2020/12/28 19:00:10 DEBUG : yt/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Attr: 
2020/12/28 19:00:10 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : bu/: ReadDirAll: 
2020/12/28 19:00:11 DEBUG : bu/: >ReadDirAll: item=3, err=<nil>
2020/12/28 19:00:11 DEBUG : bu/: Attr: 
2020/12/28 19:00:11 DEBUG : bu/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:11 DEBUG : bu/: Lookup: name="Deadliest.Catch.S16E23.Everything.Changes.1080p.AMZN.WEB-DL.DDP2.0.H.264-NTb.mkv"
2020/12/28 19:00:11 DEBUG : bu/: >Lookup: node=bu/Deadliest.Catch.S16E23.Everything.Changes.1080p.AMZN.WEB-DL.DDP2.0.H.264-NTb.mkv, err=<nil>

(I have removed the log below this)

That's what it is using.

1 Like

Do you use unraid? If yes, What drive is used by /root in unraid (asking because, I wish to physically check the drive which hosted the default rclone cache?

Destination: /.cache/rclone

As I had this error in unraid when I opened it today:

Warning: session_write_close(): write failed: No space left on device (28) in /usr/local/emhttp/login.php on line 33

Warning: session_write_close(): Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/var/lib/php) in /usr/local/emhttp/login.php on line 33

Warning: Cannot modify header information - headers already sent by (output started at /usr/local/emhttp/login.php:33) in /usr/local/emhttp/login.php on line 35

Sorry I don't use nor am I very much familiar with Unraid.

1 Like

Okay, I will take the unraid help on unraid forum, meanwhile, I wish to know one two things more:

Command: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --uid 99 --gid 100 --umask 002 --vfs-cache-mode full --vfs-cache-max-age 24h --cache-dir /mnt/disks/ua_hdd1/rclone/cache gdrive: /mnt/disks/ua_hdd1/gdrive &

  1. --vfs-cache-max-age 24h is this correct format, and How can I confirm if it is going to delete the cache after 24 hours?
  2. Do I need --buffer-size? (I don't understand the use even though I read its description)
  3. What is --dir-cache-time compared to --vfs-cache-max-age?
  4. Do I need any of these flags?
--no-checksum     Don't compare checksums on up/download.
--no-modtime      Don't read/write the modification time (can speed things up).
--no-seek         Don't allow seeking in files.
--read-only       Mount read-only.
--vfs-read-chunk-size SizeSuffix        Read the source objects in chunks. (default 128M)
--vfs-read-chunk-size-limit SizeSuffix  Max chunk doubling size (default "off")

It would be in the logs.

No

Dir cache time is documented here:

vfs-cache-max-age is documented here:

I can't say why you'd use any of those. read-only is your choice as I can't make that decision for. Do you want your mount read only? If yes, use it. If no, don't use it.

No, just use the defaults and keep your mount as simple as possible. Only change something is you have a good idea of what it does and why you are changing it.

1 Like