Guide to replaceing plexdrive/unionfs with rclone cache

That’s what I was thinking. If you turn up the Radarr logs to debug, you can get more detail.

This was the Radarr item I saw:

I’m bookmarking this topic to further study your settings, which seem to be great.

My actual setup has a cache remote inside a folder in the encrypted remote folder, and I mount the cache one. I’m doing this wrong, then?

Should I have a cache remote pointed to a media folder in drive, then encrypt the whole cache remote and mount the enccrypted remote?

I have my GD -> Cache (GD:) -> Encrypted folder inside the Cache (GD:media) for me and I mount the last item, which is the crypt.

I see.
So if I mount only the crypt remote, which refers to a folder inside the cache remote, it’s the same as if the cache remote refered to that same folder?

In other words, the caching will only occur/matter to the mounted crypted parts?

Well, I can’t see what the issue is, after bulk importing the debug logs seem normal, but Radarr spins and spins until eventually saying there are no media files in the selected location.

the debug log:
18-4-24 21:47:24.7|Debug|Parser|Movie Parsed. Casino Royale - 2006
18-4-24 21:47:24.7|Debug|Parser|Language parsed: English
18-4-24 21:47:24.7|Debug|QualityParser|Trying to parse quality for Casino Royale (2006)
18-4-24 21:47:24.7|Debug|Parser|Quality parsed: Unknown v1
18-4-24 21:47:24.7|Debug|Parser|Release Group parsed:
18-4-24 21:47:24.7|Debug|DiskScanService|Scanning ‘/home/***/***/movies/Casino Royale (2006)’ for video files
18-4-24 21:47:24.8|Debug|DiskScanService|1 video files were found in /home/***/***/movies/Casino Royale (2006)
18-4-24 21:47:24.9|Debug|ImportDecisionMaker|Analyzing 1/1 files.
18-4-24 21:47:24.9|Debug|Parser|Parsing string ‘Casino Royale (2006).mkv’
18-4-24 21:47:24.9|Debug|Parser|Movie Parsed. Casino Royale - 2006
18-4-24 21:47:24.9|Debug|Parser|Language parsed: English
18-4-24 21:47:24.9|Debug|QualityParser|Trying to parse quality for Casino Royale (2006).mkv
18-4-24 21:47:24.9|Debug|Parser|Quality parsed: HDTV-720p v1
18-4-24 21:47:24.9|Debug|Parser|Release Group parsed:
18-4-24 21:47:24.9|Debug|ImportDecisionMaker|Size: 8239430547
18-4-24 21:47:24.9|Debug|VideoFileInfoReader|Getting media info from /home/******/movies/Casino Royale (2006)/Casino Royale (2006).mkv
18-4-24 21:47:28.0|Debug|ImportDecisionMaker|Checking quality for this video file to make sure nothing mismatched.
18-4-24 21:47:28.0|Debug|ImportDecisionMaker|Checking quality for this video file to make sure nothing mismatched.
18-4-24 21:47:28.0|Debug|ImportDecisionMaker|Quality (HDTV-1080p v1) of the file is different than the one we have (HDTV-720p v1)
18-4-24 21:47:28.0|Debug|SameFileSpecification|No existing movie file, skipping
18-4-24 21:47:28.0|Debug|FreeSpaceSpecification|Skipping free space check when importing
18-4-24 21:47:28.0|Debug|SameFileSpecification|No existing movie file, skipping
18-4-24 21:47:28.0|Debug|VideoFileInfoReader|Getting media info from /home/***/***/movies/Casino Royale (2006)/Casino Royale (2006).mkv
18-4-24 21:47:28.0|Debug|FreeSpaceSpecification|Skipping free space check when importing

Even when selecting the minimum 15 at a time

The above repeats the same for each movie.

My wording was a bit poor.

For the rclone.conf

1st entry is your Google Drive which is your starting point and where you configure the client auth/etc.

2nd entry is the cache spot. The recommended method is to point to a folder inside of your Google Drive.

So mine is GD:media, which my media folder is just a normal named folder and no encryption. You can call it whatever you want as most folks have an encrypted starting point to store their media.

My 3rd entry is my encrypted mount point which points back to the type=cache (my #2 in this example) which encrypts all my media/filenames/dir names/etc.

My mount command mounts uses the 3rd entry and that is what is mounted via the rclone mount.

Can you run something like ‘mediainfo’ against the file using the same user that Radarr is running as?

yeah I get the full mediainfo detail when I run that

ps aux

chrisha+ 5655 12.4 0.0 1804236 193820 ? Sl 21:26 4:38 /usr/bin/cli /home/***/***/apps/Radarr/Radarr.exe

running as my user,

looks like one for the radarr forums.

Edit: I just deleted those carry on films and basket case and it’s fired into action again… weird as they were all dated. Radarr great for snatching, terrible for scraping…

I haven’t purged my cache after each chunk size change. I wonder if that was throwing some of the errors when I increased the size.

I added the --cache-db-purge to my systemd service start parameters just for this very reason. I figure I will restart it infrequently enough that it really shouldn’t be an issue by doing it each time the service starts.

@YipYup - The cache-db-purge doesn’t handle the chunk size changes. You can to clean the chunk directory if make changes to chunk size.

Can I ask why you have your plex info on your 2nd entry, wouldn’t you want that on your 3rd entry that plex can read?

The plex integration feature works with the cache and that’s why it goes there.

You’d see something like this in the logs to show it’s working but it is the encrypted names since I use encryption:

May 14 21:52:35 gemini rclone[3256]: smu5ej34ujbdoip1cm3mlk92q4/lprfoi8lkc2951vbhcos3sfehc/asqtmp8t491khg23v0mjn4c94uflce9fsocbks8gge6g41evbumnb2it1b3bt61b1tos1giactca6: confirmed reading by external reader
May 14 22:01:42 gemini rclone[3256]: smu5ej34ujbdoip1cm3mlk92q4/fqiaqsujuvnacu2gcicf30faedc2vvvjm0qldof71lovvqv3fnbodpb17mhm9v95jsk36g9k8rdmk/tkrhg5bv1ntslv5eub5vr4qpiq8va7n38n207v1v8m4ss8paqr46isd2pg4h8savls5c466nc7l3c: confirmed reading by external reader
May 14 22:03:20 gemini rclone[3256]: smu5ej34ujbdoip1cm3mlk92q4/lprfoi8lkc2951vbhcos3sfehc/58g2da9udu1b45mpcu0t065aghls5ri7t2s0s2tmr341cbl9dukcs8523gs30btqv7llqd2vagfde: confirmed reading by external reader

Let me clarify.

You have a gdrive, using rclone, you created a gdrive mount (normal) of your ‘media’ folder. You then created a gdrive cache mount of your ‘media’ folder. and then you created an encrypted mount of your media folder inside the cache mount?

I am a bit confused with this.

Would your first mount not have to be encrypted to hide the information from google?

Have you never hit your API limit? I seem to hit it all the time when I use Radarr/Sonarr and need to scan/update/add things from scratch. I would hope to avoid this.

I am using plexdrive and that seems to work, but the problem ends up being sometimes the API limit is breached and then I cannot use plexdrive to play due to sonarr radarr.

Any help you can provide?

I have analyze files off in both Sonarr/Radarr so I don’t use that.

My config is as follows:

[felix@gemini ~]$ cat .rclone.conf
[GD]
type = drive
client_id = client
client_secret = secret
token = {"access_token”:”token”,”token_type":"Bearer","refresh_token”:”token”,”expiry":"2018-05-19T15:03:47.495001225-04:00"}

[gcache]
type = cache
remote = GD:media
chunk_total_size = 32G
plex_url = http://192.168.1.30:32400
plex_username = username
plex_password = password
plex_token = token

[gmedia]
type = crypt
remote = gcache:
filename_encryption = standard
password = password
password2 = password
directory_name_encryption = true

Mount / Systemd

[felix@gemini ~]$ cat /etc/systemd/system/rclone.service
[Unit]
Description=RClone Service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/bin/rclone mount gmedia: /gmedia \
   --allow-other \
   --dir-cache-time=160h \
   --cache-chunk-size=10M \
   --cache-info-age=168h \
   --cache-workers=5 \
   --cache-tmp-upload-path /data/rclone_upload \
   --cache-tmp-wait-time 60m \
   --buffer-size 0M \
   --syslog \
   --umask 002 \
   --rc \
   --log-level INFO
ExecStop=/usr/bin/sudo /usr/bin/fusermount -uz /gmedia
Restart=on-abort
User=felix
Group=felix

[Install]
WantedBy=default.target

I use a GD->Cached Encrypted -> Decrypted Mount and the plex integration. You have to configured that on the cached entry by using the rclone config so you enter in the username/password and it generates the token once it connects.

Never hit an API limit in plexdrive or rclone (minus a bad config not using the cache).

Thanks for the reply.

So with analyze files off, have you had any trouble with naming files? Does this really add that much more API hits?

I think the problem is that its too many API hits so fast that it bans it for 24 hours. as I am sure i didn’t max out the APIs.

so my config file is

[gdrive-decrypt]
type = crypt
remote = /mnt/disks/plexdrive/secure
filename_encryption = standard
directory_name_encryption = true
password = <removed>
password2 = <removed>

[gdrive-secure]
type = crypt
remote = gdrive:secure
filename_encryption = standard
directory_name_encryption = true
password = <removed>
password2 = <removed>

and these are my rclone settings

rclone mount --max-read-ahead 1024k --allow-other --allow-non-empty gdrive: /mnt/disks/gdrive &
rclone mount --max-read-ahead 1024k --allow-other --allow-non-empty gdrive-secure: /mnt/disks/gdrive-secure &
plexdrive mount -c /mnt/user/appdata/plexdrive -o allow_other /mnt/disks/plexdrive/ &
rclone mount --max-read-ahead 1024k --allow-other --allow-non-empty gdrive-decrypt: /mnt/disks/gdrive-decrypt &

So as you can see I have the secure mount in the plexdrive mount to “limit” the api’s which works, but it doesn’t have write so I had to use the non plexdrive with sonarr/radarr which causes the API issues.

but I am assuming with the cache mount and the proper mounting would allow me to NOT use the plexdrive mount and hopefully speed things up.

So what would my settings be for mounting?

gcache
type = cache
remote = gdrive:
chunk_total_size = 32G
plex_url = https://localhost:32400
plex_username = username
plex_password = password
plex_token = token

[gdrive-secure]
type = crypt
remote = gdrive:secure CHANGED to gcache:
filename_encryption = standard
directory_name_encryption = true
password =
password2 =

My config and mount is just a post above.

I use GD->Cache -> Decrypted and mount the decrypted entry.

You need to configure the plex integration through rclone config to ensure it works properly and I wouldn’t paste it in.

If you don’t use the cache, you will get banned. The cache is needed if you are using GD.

1 Like

Thanks.

It made more sense while going through with it. Just did a “test” mount and I see it, but I cannot test any “limits” on it just yet.

Few questions while I am waiting.

Why do you have a “tmp” folder for rclone? (Is this stored locally where the mount is, or on the gdrive?)
Second, why do you have a umask 002 instead of 777 (or whatever it is for anyone?) or rather not have one at all?

EDIT:Also i don’t remember it asking for a PLEX token when I added name and password, can I manually add this in later?

I like to force the permissions for how I want.

002 gives it rwxrwxr-x rather than full permissions.

For Plex, you only configure the first 2 items as once it connects, it gets a token.

1 Like

Another followup. I think this is where your “tmp rclone” foider helps.

Have noticed with the cached rclone, upload through that mount is minimal to what I can be achieving. Is this a limitation set by the cache drive or what?

Only getting about 200MBit/sec or about 2-3mb/sec on a 1gbit line.
But then I can start another transfer and essentially double it.

is that where you tmp folder comes in to play?

HOw do you transfer your data, do you use sonarr/radarr to transfer it, or some other method?