Hello, I am replicating the issue in the Linux Environment (without Samba). Could you please tell me the "please share the full debug log", How to do so?
I am using -vv though.
Thanks.
Hello, I am replicating the issue in the Linux Environment (without Samba). Could you please tell me the "please share the full debug log", How to do so?
I am using -vv though.
Thanks.
Use -vv and use --log-file and share the log via pastebin/drivelink/however you want to share it.
Okay, @Animosity022 I have added the --vfs-cache-mode writes
into my command.
Command Now: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --fast-list --uid 99 --gid 100 --umask 002 -vv --vfs-cache-mode writes --log-file /mnt/disks/ua_hdd1/rclone.log gdrive: /mnt/disks/ua_hdd1/gdrive &
Steps/Process Below:
I have added the 10GB.bin
file for testing here: /mnt/disks/ua_hdd1/
Now, I am going to copy this 10GB.bin file from the above mentioned path to the rclone mount of Google Drive i.e.: /mnt/disks/ua_hdd1/gdrive
root@unraid:~# cd /mnt/disks/ua_hdd1/
root@unraid:/mnt/disks/ua_hdd1# ls
10GB.bin command.txt downloads/ gdrive/ others/ rclone.log
root@unraid:/mnt/disks/ua_hdd1# cp 10GB.bin /mnt/disks/ua_hdd1/gdrive
It is copying the file to the location right now. Stops copying around 7GB
cp: error writing '/mnt/disks/ua_hdd1/gdrive/10GB.bin': Input/output error
root@unraid:/mnt/disks/ua_hdd1#
When I do ls -la, it gives below output
root@unraid:/mnt/disks/ua_hdd1# cd gdrive
root@unraid:/mnt/disks/ua_hdd1/gdrive# ls -la
-rw-rw-r-- 1 nobody users 7512698880 Dec 27 22:18 10GB.bin
Log File: https://www.dropbox.com/s/b3eky57kvefipdh/rclone1.log?dl=0
Please check and let me know, if there is some issue, I will re-do everything. Please assist. Thanks.
You run out of space on your mount point you are copying to, so it fails:
2020/12/27 22:14:12 DEBUG : 10GB.bin(0xc0000be580): _writeAt: size=131072, off=7512653824
2020/12/27 22:14:12 DEBUG : 10GB.bin(0xc0000be580): >_writeAt: n=0, err=write /root/.cache/rclone/vfs/gdrive/10GB.bin: no space left on device
2020/12/27 22:14:12 DEBUG : &{10GB.bin (rw)}: >Write: written=0, err=write /root/.cache/rclone/vfs/gdrive/10GB.bin: no space left on device
Okay, I have actually mounted the Google Drive on a 1TB Hard Disk, which shouldn't run out of the space.
Here is the Hard Disk:
It has more than 900GB Space left.
Please assist me with the solution. Thanks.
You need to specify a location for the cache that has enough space to store a 10GB file:
2020/12/27 22:07:08 DEBUG : vfs cache: root is "/root/.cache/rclone/vfs/gdrive"
2020/12/27 22:07:08 DEBUG : vfs cache: metadata root is "/root/.cache/rclone/vfs/gdrive"
Use --cache-dir and point it wherever you want to store cache files as your /root doesn't have 10GB of space.
Okay, I have changed the cache dir to /mnt/disks/ua_hdd1/rclone/cache
Command Now: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --fast-list --uid 99 --gid 100 --umask 002 -vv --vfs-cache-mode writes --log-file /mnt/disks/ua_hdd1/rclone.log --cache-dir /mnt/disks/ua_hdd1/rclone/cache gdrive: /mnt/disks/ua_hdd1/gdrive &
I'm now going to do the test and see if it fails or works.....
Okay, It worked fine. No Error. Thanks for that. But, I have some more questions if you don't mind?
Is there anything you suggest adding (flags/commands) to this setup to improve the performance and overall robust-ness?
How can I make sure the cache director is cleaned once the work is done?
Is there any other flags which makes sure, I don't have any issue because of the cache again?
Anything else you wish to tell which I might have forgot to ask?
Thanks.
--fast-list does nothing and can be removed.
--buffer-size - any reason you are setting it? You should remove it.
You can turn off -vv once you are confident things are working. More is less. Use the defaults unless you have a reason to not use them.
It cleans itself and there is nothing you need to do. If you want to change any of the times, please check here:
Okay, I have made the changes. But regarding, Buffer Size.
My Media Consumption Process:
I download my media from Sonarr/Radarr (via. Torrent/NZB) to Local HDD and then Sonarr/Radarr copies the file directly to the Mounted Gdrive (via. rclone).
Now, I use Plex to watch the Media, so I have been using Gdrive/media/shows and Gdrive/media/movies directly onto the Plex.
So, If I play a movie, it downloads from the Google's Server directly.
Each time, when I play a media on the Plex, it loads from the Google's Server, right?
How can I make sure, I don't have to re-load the media for x-duration even if I turn off the Plex for the day?
Like, I am watching Tenet. I watched the movie till 1 Hour, but now I wish to watch next 1 hour tomorrow. But, Plex buffers the media for n number of minutes in advance for me, How can I make sure, I retain those n number of minutes for x duration? So that, when I re-launch/resume the Plex, it don't have to redownload?
I hope I am able to explain this.
Thanks.
Read through vfs-cache-mode full and most folks use that:
Okay, --vfs-cache-mode full
is the thing I should use instead of writes. But then, How do I set the duration/age of the cache?
--vfs-cache-max-age duration
Is this something I need to use for that purpose?
Also, Where is this saving? Which location?
Thanks.
Like 3 posts up, you set the location of the cached files with --cache-dir.
Use whatever duration you'd like to keep as it's really up to you.
Hello, I'm facing another issue. Since, I am using --vfs-cache-mode full
now, I think it's trying to cache everything without even running the file?
Because of which, /root/.cache/rclone
is taking up all the space, causing a lot of errors to my unraid.
Command: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --uid 99 --gid 100 --umask 002 --vfs-cache-mode full --vfs-cache-max-age 24h --cache-dir /mnt/disks/ua_hdd1/rclone/cache gdrive: /mnt/disks/ua_hdd1/gdrive &
Please Check, Because it is not even writing the cache on the given directory/folder. But instead still going onto the /root/.cache/rclone
2020/12/28 18:43:19 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:19 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo(0xc002c0d240): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E01 - Let the Titan Games Begin - Trials 1 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:21 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:21 ERROR : media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo(0xc002c0db80): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Titan Games/Season 01/The Titan Games - S01E06 - The Titan Games Trials 6 [WEBDL-1080p x264 EAC3].nfo: no such file or directory
2020/12/28 18:43:27 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:27 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo(0xc0013b0b40): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-28 - Ken Jeong, Mayor Pete Buttigieg, Sam Hunt WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:28 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:28 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo(0xc0013b11c0): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-10-30 - Nick Offerman, Stacey Abrams, Busta Rhymes ft. Anderson .Paak WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:29 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:29 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo(0xc0013b1540): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-06 - Milo Ventimiglia, Marc Maron, Oneohtrix Point Never WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:30 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo: vfs cache: item close failed: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo: no such file or directory
2020/12/28 18:43:30 ERROR : media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo(0xc0013b1980): RWFileHandle.Release error: vfs cache item: failed to write metadata: open /.cache/rclone/vfsMeta/gdrive/media_swizzin/sonarr/The Tonight Show Starring Jimmy Fallon/Season 08/The Tonight Show Starring Jimmy Fallon - 2020-11-30 - Nick Kroll, Lily Collins, Matt Berninger WEBDL-1080p.nfo: no
Please check the top of your log with debug and it'll tell you where it's writing to.
Thanks for the reply. How do I confirm if it is using the cache-dir as I have mentioned, myself and not trusting the log?
Currently, it is showing this:
root@unraid:/mnt/disks/ua_hdd1# cat rclone.log
2020/12/28 19:00:01 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "mount" "--allow-other" "--buffer-size" "256M" "--dir-cache-time" "96h" "--timeout" "1h" "--default-permissions" "--uid" "99" "--gid" "100" "--umask" "002" "--vfs-cache-mode" "full" "--vfs-cache-max-age" "24h" "--cache-dir" "/mnt/disks/ua_hdd1/rclone/cache" "-vv" "--log-file" "/mnt/disks/ua_hdd1/rclone.log" "gdrive:" "/mnt/disks/ua_hdd1/gdrive"]
2020/12/28 19:00:01 DEBUG : Creating backend with remote "gdrive:"
2020/12/28 19:00:01 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2020/12/28 19:00:02 DEBUG : vfs cache: root is "/mnt/disks/ua_hdd1/rclone/cache/vfs/gdrive"
2020/12/28 19:00:02 DEBUG : vfs cache: metadata root is "/mnt/disks/ua_hdd1/rclone/cache/vfs/gdrive"
2020/12/28 19:00:02 DEBUG : Creating backend with remote "/mnt/disks/ua_hdd1/rclone/cache/vfs/gdrive"
2020/12/28 19:00:02 DEBUG : Google drive root '': Mounting on "/mnt/disks/ua_hdd1/gdrive"
2020/12/28 19:00:02 INFO : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2020/12/28 19:00:02 DEBUG : : Root:
2020/12/28 19:00:02 DEBUG : : >Root: node=/, err=<nil>
2020/12/28 19:00:02 DEBUG : /: Attr:
2020/12/28 19:00:02 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Attr:
2020/12/28 19:00:10 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: ReadDirAll:
2020/12/28 19:00:10 DEBUG : /: >ReadDirAll: item=10, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Attr:
2020/12/28 19:00:10 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="P"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=P/, err=<nil>
2020/12/28 19:00:10 DEBUG : P/: Attr:
2020/12/28 19:00:10 DEBUG : P/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="Test1"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=Test1/, err=<nil>
2020/12/28 19:00:10 DEBUG : Test1/: Attr:
2020/12/28 19:00:10 DEBUG : Test1/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="bu"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=bu/, err=<nil>
2020/12/28 19:00:10 DEBUG : bu/: Attr:
2020/12/28 19:00:10 DEBUG : bu/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="media"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=media/, err=<nil>
2020/12/28 19:00:10 DEBUG : media/: Attr:
2020/12/28 19:00:10 DEBUG : media/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="media_swizzin"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=media_swizzin/, err=<nil>
2020/12/28 19:00:10 DEBUG : media_swizzin/: Attr:
2020/12/28 19:00:10 DEBUG : media_swizzin/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="nzb"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=nzb/, err=<nil>
2020/12/28 19:00:10 DEBUG : nzb/: Attr:
2020/12/28 19:00:10 DEBUG : nzb/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="test"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=test/, err=<nil>
2020/12/28 19:00:10 DEBUG : test/: Attr:
2020/12/28 19:00:10 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="torrent"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=torrent/, err=<nil>
2020/12/28 19:00:10 DEBUG : torrent/: Attr:
2020/12/28 19:00:10 DEBUG : torrent/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="youtube"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=youtube/, err=<nil>
2020/12/28 19:00:10 DEBUG : youtube/: Attr:
2020/12/28 19:00:10 DEBUG : youtube/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Lookup: name="yt"
2020/12/28 19:00:10 DEBUG : /: >Lookup: node=yt/, err=<nil>
2020/12/28 19:00:10 DEBUG : yt/: Attr:
2020/12/28 19:00:10 DEBUG : yt/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : /: Attr:
2020/12/28 19:00:10 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:10 DEBUG : bu/: ReadDirAll:
2020/12/28 19:00:11 DEBUG : bu/: >ReadDirAll: item=3, err=<nil>
2020/12/28 19:00:11 DEBUG : bu/: Attr:
2020/12/28 19:00:11 DEBUG : bu/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2020/12/28 19:00:11 DEBUG : bu/: Lookup: name="Deadliest.Catch.S16E23.Everything.Changes.1080p.AMZN.WEB-DL.DDP2.0.H.264-NTb.mkv"
2020/12/28 19:00:11 DEBUG : bu/: >Lookup: node=bu/Deadliest.Catch.S16E23.Everything.Changes.1080p.AMZN.WEB-DL.DDP2.0.H.264-NTb.mkv, err=<nil>
(I have removed the log below this)
That's what it is using.
Do you use unraid? If yes, What drive is used by /root
in unraid (asking because, I wish to physically check the drive which hosted the default rclone cache?
Destination: /.cache/rclone
As I had this error in unraid when I opened it today:
Warning: session_write_close(): write failed: No space left on device (28) in /usr/local/emhttp/login.php on line 33
Warning: session_write_close(): Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/var/lib/php) in /usr/local/emhttp/login.php on line 33
Warning: Cannot modify header information - headers already sent by (output started at /usr/local/emhttp/login.php:33) in /usr/local/emhttp/login.php on line 35
Sorry I don't use nor am I very much familiar with Unraid.
Okay, I will take the unraid help on unraid forum, meanwhile, I wish to know one two things more:
Command: rclone mount --allow-other --buffer-size 256M --dir-cache-time 96h --timeout 1h --default-permissions --uid 99 --gid 100 --umask 002 --vfs-cache-mode full --vfs-cache-max-age 24h --cache-dir /mnt/disks/ua_hdd1/rclone/cache gdrive: /mnt/disks/ua_hdd1/gdrive &
--vfs-cache-max-age 24h
is this correct format, and How can I confirm if it is going to delete the cache after 24 hours?--buffer-size
? (I don't understand the use even though I read its description)--dir-cache-time
compared to --vfs-cache-max-age
?--no-checksum Don't compare checksums on up/download.
--no-modtime Don't read/write the modification time (can speed things up).
--no-seek Don't allow seeking in files.
--read-only Mount read-only.
--vfs-read-chunk-size SizeSuffix Read the source objects in chunks. (default 128M)
--vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default "off")
It would be in the logs.
No
Dir cache time is documented here:
vfs-cache-max-age is documented here:
I can't say why you'd use any of those. read-only is your choice as I can't make that decision for. Do you want your mount read only? If yes, use it. If no, don't use it.
No, just use the defaults and keep your mount as simple as possible. Only change something is you have a good idea of what it does and why you are changing it.