Something like this would probably be easier with mergerfs. MergerFS can have the mounts changed without umount’ing[0]
I had worked with someone to make mergerfs resilient to underlying filesystem errors. The ‘read-failover’ branch implemented a feature to allow a read failure on one drive to failover to another. The problem was that rclone continues to show files even when the mount is for all practical purposes dead. Ideally the files/directories would disappear (like when a harddrive fails) so the other drives may be picked up. I’ve yet to request the feature in rclone though. I’m in the process of refactoring some of mergerfs’ code which would allow handling of underlying errors better. For example: where an open would fail (like it does with rclone today) for the file the policy said to open but mergerfs would continue searching for another file on other drives till it finds one that works.
This all seems like more hassle than its worth. If you are using Google drive, my recommendation is still to use node-gdrive-fuse as your RO mount point, local storage to download media from sonarr/radarr/couchpotato, and rclone copy/move to upload the files to your google drive library. Scans by plex on node-gdrive-fuse do not cause a ban.
Unfortunately I am limited by my storage of choice, namely unraid. I know many who are in similar predicaments, so if i can provide a method for them, I’d like to.
Edit: Found a loop condition. Changed it to prevent.
This is my most recent script, given the great input I’ve received in this pursuit:
#!/bin/bash
#This script will monitor the mount's LogFile, and switch the ACDSecure drive and the GDriveSecure drive in the UnionFS Mount when there is a "403 Forbidden" error.
#
#Define your mount points & logfile here:
#
GDriveSecure="/mnt/disks/GDriveSecure"
ACDSecure="/mnt/disks/ACDSecure"
UnionFS="/mnt/disks/UnionFS"
StagingFolder="/mnt/user/Media/Staging"
LOGFILE="/mnt/user/appdata/PlexMediaServer/mount.log"
PMSAppData="/mnt/user/appdata/PlexMediaServer"
tail -fn0 $LOGFILE | \
while read line ; do
echo "$line" | grep "403 Forbidden" # Could probably be a bit more specific, does this always imply ban?
if [ $? = 0 ] ; then
sleep 5
#This script will test your mount points and make sure they are remounted if they should fail. If one fails, it will switch the UnionFS to the other, but not modify should the other be okay.
#Begin GDriveSecure Test
if [[ -f "$GDriveSecure/GDriveSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, GDriveSecure mounted" | tee -a "/var/log/acd-check.log"
else
echo "$(date "+%d.%m.%Y %T") ERROR: GDriveSecure not mounted remount in progress" | tee -a "/var/log/acd-check.log"
if [[ -f "$UnionFS/ACDSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ACDSecure mounted to UnionFS" | tee -a "/var/log/acd-check.log"
else
echo "$(date "+%d.%m.%Y %T") ERROR: ACDSecure not mounted to UnionFS. Remount in progress" | tee -a "/var/log/acd-check.log"
fusermount -u $UnionFS
fusermount -uz $UnionFS
umount -l $UnionFS
umount -f $UnionFS
mkdir -p $UnionFS
sleep 2
unionfs \
-o cow,allow_other,direct_io,auto_cache,sync_read,nonempty,hide_meta_files \
$StagingFolder=RW:$ACDSecure=RO \
$UnionFS
sleep 2
if [[ -f "$UnionFS/ACDSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: ACDSecure Remount to UnionFS successful" | tee -a "/var/log/acd-check.log"
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: ACDSecure Remount to UnionFS failed." | tee -a "/var/log/acd-check.log"
fi
#This will restore any items that may have accidentally gotten trashed during the last mount failure.
sqlite3 $PMSAppData/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/com.plexapp.plugins.library.db "UPDATE metadata_items set deleted_at=null"
fi
fusermount -u $GDriveSecure
fusermount -uz $GDriveSecure
umount -l $GDriveSecure
umount -f $GDriveSecure
sleep 2
rclone mount \
--allow-non-empty \
--allow-other \
--dir-cache-time 60m \
--max-read-ahead 4G \
--buffer-size 1G \
--contimeout 15s \
--low-level-retries 1 \
--no-check-certificate \
--quiet \
--stats 0 \
--retries 10 \
--timeout 10s \
--log-file=/mnt/user/appdata/PlexMediaServer/mount.log \
GDriveSecure:/ $GDriveSecure &
sleep 2
if [[ -f "$GDriveSecure/GDriveSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: GDriveSecure Remount successful" | tee -a "/var/log/acd-check.log"
mkdir -p $UnionFS
sleep 2
unionfs \
-o cow,allow_other,direct_io,auto_cache,sync_read,nonempty,hide_meta_files \
$StagingFolder=RW:$GDriveSecure=RO \
$UnionFS
#This will restore any items that may have accidentally gotten trashed during the last mount failure.
sqlite3 $PMSAppData/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/com.plexapp.plugins.library.db "UPDATE metadata_items set deleted_at=null"
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: GDriveSecure Remount failed." | tee -a "/var/log/acd-check.log"
fi
fi
#
#Begin ACDSecure Test
#
if [[ -f "$ACDSecure/ACDSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, ACDSecure drive mounted" | tee -a "/var/log/acd-check.log"
else
echo "$(date "+%d.%m.%Y %T") ERROR: ACDSecure Drive not mounted remount in progress" | tee -a "/var/log/acd-check.log"
if [[ -f "$UnionFS/GDriveSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, GDriveSecure mounted to UnionFS" | tee -a "/var/log/acd-check.log"
else
echo "$(date "+%d.%m.%Y %T") ERROR: GDriveSecure not mounted to UnionFS. Remount in progress" | tee -a "/var/log/acd-check.log"
fusermount -u $UnionFS
fusermount -uz $UnionFS
umount -l $UnionFS
umount -f $UnionFS
mkdir -p $UnionFS
sleep 2
unionfs \
-o cow,allow_other,direct_io,auto_cache,sync_read,nonempty,hide_meta_files \
$StagingFolder=RW:$GDriveSecure=RO \
$UnionFS
sleep 2
if [[ -f "$UnionFS/GDriveSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: GDriveSecure Remount to UnionFS successful" | tee -a "/var/log/acd-check.log"
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: GDriveSecure Remount to UnionFS failed." | tee -a "/var/log/acd-check.log"
fi
#This will restore any items that may have accidentally gotten trashed during the last mount failure.
sqlite3 $PMSAppData/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/com.plexapp.plugins.library.db "UPDATE metadata_items set deleted_at=null"
fi
fusermount -u $ACDSecure
fusermount -uz $ACDSecure
umount -l $ACDSecure
umount -f $ACDSecure
sleep 2
rclone mount \
--allow-non-empty \
--allow-other \
--dir-cache-time 60m \
--acd-templink-threshold 0 \
--max-read-ahead 4G \
--buffer-size 1G \
--contimeout 15s \
--low-level-retries 1 \
--no-check-certificate \
--quiet \
--stats 0 \
--retries 10 \
--timeout 10s \
--log-file=/mnt/user/appdata/PlexMediaServer/mount.log \
ACDSecure:/ $ACDSecure &
sleep 2
if [[ -f "$ACDSecure/ACDSecureTest.txt" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: ACDSecure Remount successful" | tee -a "/var/log/acd-check.log"
mkdir -p $UnionFS
sleep 2
unionfs \
-o cow,allow_other,direct_io,auto_cache,sync_read,nonempty,hide_meta_files \
$StagingFolder=RW:$ACDSecure=RO \
$UnionFS
#This will restore any items that may have accidentally gotten trashed during the last mount failure.
sqlite3 $PMSAppData/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/com.plexapp.plugins.library.db "UPDATE metadata_items set deleted_at=null"
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: ACDSecure Remount failed." | tee -a "/var/log/acd-check.log"
fi
fi
echo "The UnionFS Switch has been completed."
#This moves the log file so that it the script does not keep looping
mv $LOGFILE "$LOGFILE $(date)"
fi
done
might affect your GDrive, once it’s banned media will not play on Plex Cloud. At one point during all this I did believe Plex Cloud was an issue, so much so that I removed it completely, but it may have just been me clutching at straws
Yes, I have been attempting to do similarly: it watches the log file and when an error is detected, it runs a check on all the mounts, attempts to remount, if remounting is successful, then it remounts the unionfs and then restarts plex. If it fails to remount, and open a file, reading its contents, then it fails and unmounts the drive fully. I’m still getting the code right, but it looks promising. When the failure happens, the unmount happens, causing the unionfs to default to the next on the list. Due to the additional file contents check, if it was just a one off error, it just continues on. I had been trying to consolidate my scripts into single scripts rather than chaining function built scripts. I’m now trying to break out my scripts into functions to call. Keeping everything as single scripts was too difficult, so building out functions in separate scripts is much easier.
And then I’ve also been testing gdriveocamlfuse. Looks very good. Like extremely so. But it crashes on unraid almost every hour. Not really reliable for unraid (Slackware based). Once I have that working, I’ll post here too. I’ve got an ubuntu vm I’m building to see how that works. So far, it’s surpassed the one hour mark of being mounted, but then again, it doesn’t have any traffic on it yet. If this works, I may pull a small arch vm together to host all my cloud storage. When I get that far though, I may want to do a docker. We’ll see.
I think I almost have enough here to write a thesis on cloud storage.
A minor update:
I’ve made a Ubuntu 16.04 server, and am currently running Google-Drive-Ocamlfuse there. I then use a local rclone mount, which I then share via a SMB connection to this, and it seems to be holding very well. I’ve placed a check and remount script into the cron for a 5 minute check, which seems to be working very well at keeping the mount mounted. Next I have this joined with UnionFS-fuse to make a very solid system.
My typical time to playing a video via remote web interface: ~5 seconds.
My typical time to skip to another location in the video: 2-7 seconds, depending on if it has already been cached by the mount.
Lag time to play on that have been already remux’d (read, very small) are almost imperceptible to that of my local NAS.
I’ll post all of my scripts once I’m sure it’s running well. I think my issues with everything have been the fact that I’ve been trying to get everything to run on UnRaid, rather than just doing a VM.
Gave up with Google-Drive-Ocamlfuse… It’s just not stable enough. The mount drops almost every time a video is played.
Now I’m trying out PlexDrive. Seems to be doing very well, except for actually responding to directory structure listings, which the author has said may be resolved in the next release. I’m prepping for that release.
I’m running Ubuntu 16.04 and haven’t had any issues with the mount dropping. I’ve had 6 streams going as my high without any issues. If you are seeing drops, best to report the issue and see what it is so it can get fixed.
