I’m struggling to the get the grip of the file/folder flow utilising rclone cache in my media server. I have some basic questions just to understand the fundamentals of the flow:
I’ve mounted a unencrypted rclone cache remote (gcache) to /media/. Gcache is pointing to my google drive mount (gdrive). In the /media folder I can see my movies and tv folders as expected.
I’ve setup Plex library to look in the /media mount. Works as expected.
Now the big questions. What do I do with radarr/sonarr? Can they do file/folder renaming and moves directly to the cached /media mount? Or do I have to use the unionfs local/cloud trick still? Cache mounts are RW right?
If I make a script that moves local files from /media to gdrive every once in a while, will rclone manage to differ between a cached entry and a local file?
Sorry for my stupidness. It’s really hard to find good information regarding rclone cache as it’s still quite new. And I’m the kind of guy that needs guides to learn
So if I understand you correctly:
All local files that are moved to my mounted /media/TV or Movies folders are automatically uploaded to gdrive and will therefore not use any space on the local drive?
Yeah, default is to move immediately. This can be tuned with the --cache-tmp-upload-path and --cache-tmp-wait-time. Basically --cache-tmp-upload-path stores the added files on your local drive for an amount of time specified by --cache-tmp-wait-time, after which it is uploaded to GDrive. I have set this time to 60 minutes in my config to ensure that even large files finish copying and also to account for immediate repacks etc. You can set this according to your server configuration.
I’ve totally been over complicating how this works. Guess not coming from a plexdrive and unionfs setup, and then reading about the whole bunch, just made me mix it all togheter.
It’s not you. I still deploy rclone cache as a read only and deploy unionfs on top to transfer due to our own movement scripts for better handling and speeds. Plus I don’t end up with a bottle neck cache. Depends on your use.
Having it read-only reduces the ability to easily upgrade and delete files. Unionfs adds complexity along with another script.
By using just rclone cache (not sure what the bottleneck you are talking about) and the cache-tmp-upload, it simplifies things quite a bit and allows all the software that was mentioned to work as-is without doing anything else.
I point everything to my decrypted /gmedia/TV or /gmedia/Movies mount and store stuff locally for 60 minutes before uploading. I personally also use plex_autoscan because that helps me with replacing media if I upgrade as it handles emptying the trash and has a failsafe to not empty if more the ‘x’ items in the trash.
@Fjesnes You can use the config provided by @Animosity022. It is the base of my config too. There are a few tweaks in relation to the wait-time & workers but that will have to be tuned anyway for your system.
I run both sonarr & radarr on the same machine as plex, so I don’t use plex_autoscan to handle the update and instead just let sonarr & radarr notify plex as needed. I also do not have any script for emptying the trash and do it manually when needed. This allows me to maintain two versions of a specific movie if needed. It shouldn’t matter much anyway because GSuite has unlimited storage.
Easily? Not really. You can see the files locally but they are encrypted so you have to translate them back.
You can see in the logs when it uploads and moves stuff:
May 8 06:44:29 gemini rclone[3092]: smu5ej34ujbdoip1cm3mlk92q4/q42c44qbkijmuh0ts7efctb39rrqjpp4knu806tmvpst0lvqajo0/gld67r8m7tg17as5lhpk5u7doh58l33g0dff18mb1fl0vlr0mg1c5bu0ap884575ena4e88uh7kva: background upload: started upload
May 8 06:46:00 gemini rclone[3092]: smu5ej34ujbdoip1cm3mlk92q4/q42c44qbkijmuh0ts7efctb39rrqjpp4knu806tmvpst0lvqajo0/gld67r8m7tg17as5lhpk5u7doh58l33g0dff18mb1fl0vlr0mg1c5bu0ap884575ena4e88uh7kva: Copied (new)
May 8 06:46:00 gemini rclone[3092]: smu5ej34ujbdoip1cm3mlk92q4/q42c44qbkijmuh0ts7efctb39rrqjpp4knu806tmvpst0lvqajo0/gld67r8m7tg17as5lhpk5u7doh58l33g0dff18mb1fl0vlr0mg1c5bu0ap884575ena4e88uh7kva: Deleted
May 8 06:46:00 gemini rclone[3092]: smu5ej34ujbdoip1cm3mlk92q4/q42c44qbkijmuh0ts7efctb39rrqjpp4knu806tmvpst0lvqajo0/gld67r8m7tg17as5lhpk5u7doh58l33g0dff18mb1fl0vlr0mg1c5bu0ap884575ena4e88uh7kva: background upload: uploaded entry
May 8 06:46:00 gemini rclone[3092]: smu5ej34ujbdoip1cm3mlk92q4/q42c44qbkijmuh0ts7efctb39rrqjpp4knu806tmvpst0lvqajo0/gld67r8m7tg17as5lhpk5u7doh58l33g0dff18mb1fl0vlr0mg1c5bu0ap884575ena4e88uh7kva: finished background upload
I honestly just check every so often to see if any files are there and it doesn’t clean up the directories yet, so there is some garbage laying around but only a few MB.
If you are using --syslog, it’ll be in /var/log/syslog, but I might recommend to remove the --syslog and just use the --log-file and put in the location you want.
I decided to change the --cache-tmp-upload-path from /data/gcache to /opt/gcache. I just changed it in the mount command in rclone.service and rebooted. Now I see a lot of errors like this in the log:
2018/05/10 12:00:57 ERROR : TV/The SHOWNAME/Season 1/SHOWNAME- S01E10 - Night Bluray-720p.mkv: error refreshing object in : in cache fs Google drive root 'Media': object not found