Hi there!
First off, @ncw thank you for Rclone, and thanks to all the contributors.
I’ve spent the last week setting up a new PLEX server with Google Drive for storage using Rclone. I documented my set-up as I went and thought I’d post it in case it helps anyone else and for any feedback on how I might optimize anything. (or just so anyone can point out any noob mistakes I’m making)
Overview
Use Case:
I have a NUC (Windows 10), running Radarr, Sonarr and PLEX Server. NAS For local back up and Media Storage. Offsite back-up with Crashplan.
Want to add cloud storage (Gdrive) to meet two needs:
- Additional storage capacity/offsite backup for media
- Offsite backup for NUC and other devices (to replace Crashplan)
Additional Goal:
I have movies/TV in my collection that would be difficult to replace. I envisage most media in the cloud, but retaining a local copy of that “important” collection. All of that will be seamlessly presented to PLEX.
High Level workflows
Media:
Sonarr / Radarr run on the NUC and download to the NAS. Overnight scripts will copy to Gdrive.
A Rclone union will present PLEX with a single drive containing the onsite/gdrive media.
I’ll manage local storage capacity manually (for now). i.e. I’ll manually delete local stuff (I don’t want a local backup of) as and when.
Backup:
I use Syncredible to backup files/Windows Image from Devices to the NAS (Everything backed up weekly, but staged over a full week. i.e.backup is broken into seven parts and one part is done each night) overnight scripts will sync to Gdrive
RClone Config, In my case is something lilke:
Drive Remote - Gdrive:
Crypt Remote Media_vfs: (for my Media storage)
Crypt Remote Backup_vfs: (for my Backups storage)
Union Media_Union (Union of Media_vfs: and my local media storage)
Disk / Drive view:
Media / Backup workflows Overview:
Instructions
Set up Rclone with VFS
Create ClientID on GoogleDrive:
Create a client ID on GoogleDrive API. I followed the guidance here: https://www.iperiusbackup.net/en/how-to-enable-google-drive-api-and-get-client-credentials/
Install and set up Rclone:
- Create C:\Program Files\rclone.
- Download rclone from http://rclone.org/downloads/ and unzip it to C:\Program Files\rclone
Configure Rclone:
There are lots of other excellent guides that step throug the config.I won’t do step by step here. My Config file ended up like…
[Gdrive]
type = drive
client_id = *redacted*.apps.googleusercontent.com
client_secret = *redacted*
scope = drive
token = *redacted*
[Media_vfs]
type = crypt
remote = Gdrive:Media_Storage_crypt
filename_encryption = standard
directory_name_encryption = true
password = *redacted*
[Backup_vfs]
type = crypt
remote = Gdrive:Backup_Storage_crypt
filename_encryption = standard
directory_name_encryption = true
password = *redacted*
[Media_union]
type = union
remotes = Media_vfs: V:\
Notes:
- Only the last remote is used to write to and delete from, all other remotes are read-only.
- The V: Drive here should be F:. F: is the drive my local media is mounted on (my NAS). BUT I’m going to start the UNION under a service. The service uses the SYSTEM Account and F: is only visible from my account (thanks Windows!). I will explain how to create the drive V: later on.
Automate the union mounting using NSSM.
Great. I have my remotes. I want them to Mount two of them (Backup_vfs: & Media_Union:) at Boot up. I’m going to use NSSM for that.
General Guidance here:
https://nssm.cc/usage
https://forum.rclone.org/t/how-to-propery-setup-rclone-mount-on-windows-using-nssm-as-a-service/7055
& https://forum.rclone.org/t/installing-rclone-mount-on-windows-as-service/4649
How to Run NSSM:
Copy nssm.exe to same directory that rclone is in. Launch NSSM, by opening an elevated command line. (Type cmd in the search bar and right click and choose “run as administrator.”). Then type
- cd c:\rclone (or wherever you have rclone located)
- nssm install “Service Name Here”
- Fill in the info below
- Still in the command prompt, type
services.msc - Scroll down and choose your service and right click and start it.
IF it fails to start, stop the service.
Go back to command prompt and run sc.exe delete “Service Name Here”
Enter the command from step 2 above and repeat until you complete everything with no typos and it works.
NSSM settings:
NSSM Settings Backup_vfs:
Service Name =Rclone Backup_vfs Mount Google Drive
Application tab:
Path:c:\Program Files\rclone\rclone.exe
Startup Directory:c:\Program Files\rc- lone
Arguments:mount Backup_vfs: --vfs-cache-mode writes --dir-cache-time 72h N: --config "C:\Users\robev\.config\rclone\rclone.- conf"
Service name:Rclone Backup_vfs Mount Google Drive
Details tab:
Display name:Rclone Backup_vfs Mount Google Drive
Description:Automatically mounts the encrypted google drive using rclone. Encrypted drive is Backup_vfs local drive is N:
Exit actions tab:
Delay resart by:10000ms (10 seconds)
NSSM Settings Media_Union:
Service Name =Rclone Media_Union Mount Google Drive
Application tab:
Path:c:\Program Files\rclone\rclone.exe
Startup Directory:c:\Program Files\rc- lone
Arguments:mount --allow-other --dir-cache-time 72h --drive-chunk-size 64M --log-level INFO --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit off Media_union: H: --config "C:\Users\robev\.config\rclone\rclone.conf" --vfs-cache-mode writes
Service name:Rclone Media_Union Mount Google Drive
Details tab:
Display name:Rclone Media_Union Mount Google Drive
Description:Automatically mounts the encrypted google drive using rclone. Encrypted drive is Media_union local drive is H:
Exit actions tab:
Delay resart by:10000ms (10 seconds)
Note: To change the parameters of NSSM / rclone after creation of the service, use regedit.exe to edit the registry:
\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\RCLONE SERVICE NAME\Parameters
Creating the V: drive (for the Media_Union Remote)
Why do I need to create a V: drive? What I want is a union Media_vfs: and my Local Media storage (f:). BUT I’ve set up Media_Union H: to start as a service (using NSSM above).
Problem, in my use case, is the local drive, (f:) is a share drive. Share drives are only visible to the user who creates them (in this case my user account). However, the service, I’ve created, uses the SYSTEM Account so F: is NOT visible to it. (the service still creates Media_Union H: for me, but only the gdrive content is visible!
The workaround for this …
Create, another mapping of my Local Media storage (f:). In this case as drive V:
Specifically use task schedular to create V: on startup, under the SYSTEM user, and will be visible to my Media_Union.
Open cmd (with admin privs) amd type :
schtasks /create /tn "my_mount" /tr "net use V: \hostname\directory /persistent:yes" /sc onstart /RU SYSTEM /RL HIGHESTFollowed by a restart of your server
• or by executing: schtasks /run /tn “my_mount”
• or going to Task Scheduler and starting the task from there.
• IF your share needs credentials add “/user:device\username Password
• IF your share is mounted already Windows may tell you you can’t have two users use the same credentials. It did for me and I got around it by using the device IP rather than the name.Example:
schtasks /create /tn "my_mount" /tr “net use V: \\192.168.1.99\SeagateNAS\Storage /persistent:yes /user:PERSONALCLOUD\username Password /sc onstart /RU SYSTEM /RL HIGHEST
Notes:
The workaround shows the V: drive in My Computer (This PC). BUT as a “disconnected Network Drive”. However, it’s not caused me problems other than aesthetics.Note: IF there are problems with the service running before the task at boot up. I had problems on 1 reboot, but all the rest have been fine. But still try the advice here: https://stackoverflow.com/questions/30331462/boot-order-on-windows-services-vs-scheduled-tasks
At this point I can re-point PLEX at the drive H: (Media_Union) and Sonarr/Radarr at the same. (following all the good practice on repointing PLEX over on the PLEX forums)
Setting up automated copy/Move/Syncs for Rclone
I’ve not set up the back-up scripts (yet) but intend to use the following commands:
Copy Media from NAS F: (storage) to Gdrive G: (Media_vfs)
"C:\Program Files\rclone\rclone.exe" --config C:/Users/robev/.config/rclone/rclone.conf copy --verbose --transfers 4 --checkers 8 --log-file="C:\Users\robev.config\rclone\Rclone_Log_Files\Media_vfs.txt" --fast-list --max-backlog=100000 --tpslimit=10 \PERSONALCLOUD\Storage Media_vfs: --dry-run
Sync (backup) from NAS M: (Backup) to Gdrive N: (Backup_vfs)
"C:\Program Files\rclone\rclone.exe" --config C:/Users/robev/.config/rclone/rclone.conf sync --verbose --transfers 6 --checkers 8 --log-file=C:\Users\robev.config\rclone\Rclone_Log_Files\Backup_vfs.txt --fast-list --max-backlog=100000 --tpslimit=10 \personalcloud\Backup Backup_vfs:NUC_Backup --exclude "/WindowsImageBackup/**" --dry-run
remember to run with --dry-run first and check all is good before running properly!!
Edits:
- Updated Copy and Sync commands.
- Updated Union mount parameters.