rClone VFS with Cache Remote + cache-tmp-upload-path

Second time around configuring Plex Gdrive. On the advice from someone else, I adapted this mount command.

rclone mount --vfs-cache-mode writes --allow-other rcrypt:/ /mnt/rcrypt/

I have slow upload so need to keep files locally (and available to Plex) as they upload to the cloud. From my previous attempt, I use a drive, crypt and cache remote. I ended up with:

rclone mount --vfs-cache-mode writes --allow-other --cache-tmp-upload-path /mnt/synology/media/gd-tmp gd-crypt:/ /mnt/PlexRemote/

It plays back fine but is really slow for Plex scanning.

So you are using just the crypt mount now?

You'd want to make sure your own client ID/API key is setup:

https://rclone.org/drive/#making-your-own-client-id

You should also add --dir-cache-time to your mount command as the default is a very low value.

I use 96 hours for mine:

--dir-cache-time 96h

Other than that, the defaults are good. Scanning would be slow the first time and should be fast after that.

1 Like

Just got an API key.

I still have a cache remote. Drive, Cache + Crypt = Mount (VFS settings + cache-tmp-upload settings.)

I should just have drive and crypt with only VFS settings in the mount, shouldn't I?

If you are using the plex integration with cache, scans are slow as it only uses 1 worker for that.

Increasing dir-cache-time and the info_age would help.

Not using plex integration. Using your dir-cache-time atm. If I add info_age should I remove --vfs-cache-mode?

The terminology is definitely a bit mixed so apologize for that.

vfs-cache-mode writes tends to be used if you are seeing an error message in the logs telling you to turn it on. It's not part of the cache backend.

You can remove it and if you aren't see any issues, should be fine.

You definitely do want to increase the caching times as that speeds things up, but:

To avoid getting in a scenario where dir cache has obsolete data and cache would have the correct one, try to set --dir-cache-time to a lower time than --cache-info-age . Default values are already configured in this way.

So if dir-cache-time is 96 hours, cache-info-age would need to be bigger so 100 hours or even 97 would be fine.

Uninstalled rclone and reinstalled it and added my remotes, added my own API key and included those commands in a mount command.

I have 80-ish GB in GDrive (looking at the web interface) with folder structure encrypted/tv/TVShowName/Season/File but in my mounted GDrive folder I only see the tv folder.

Can you share what your rclone.conf is minus the keys/passwords and what your mount command is?

[gdrive]
type = drive
client_id = My own
client_secret = My own
scope = drive
token = {"access_token":"XYZ")

[gd-cache]
type = cache
remote = gdrive:
chunk_size = 10M
info_age = 3h0m0s
chunk_total_size = 50G

[gd-crypt]
type = crypt
remote = gd-cache:/encrypted
filename_encryption = standard
directory_name_encryption = true
password =Very long

Latest mount command: rclone mount --dir-cache-time 96h --cache-info-age 100h --allow-other --allow-non-empty --cache-tmp-upload-path /mnt/synology/media/gd-tmp gd-crypt: /mnt/PlexRemote

I'd take the info_age out of your rclone.conf if you are doing it from the command line so it doesn't get confusing.

I'd remove non-empty as that allows for mounting over things and hiding stuff. Generally bad things happen.

If you stop the mount as you can only do one thing on the cache backend at a time, and check:

rclone lsd gd-crypt:

are you seeing the high level directories and does everything look ok?
rclone version is latest stable at 1.47?

You can also stop it and try to remove the cache.db files or mount it with --cache-db-purge to clean up everything and make try that as well.

I get the below line from the rclone lsd gd-crypt:

-1 2019-05-28 18:32:50 -1 tv

Yeah, using rclone v1.47.0.

Should it take a long time for files to be visible in the mounted folder once rclone mount is run?

The normal polling interval is 1 minute so at most, you'd wait a minute for something to appear if uploaded it from somewhere else.

So if you mount it, you can see the tv folder, but nothing under it and in the webui you are seeing encrypted folder names and such?

If so, we can do some testing without cache as you can make another entry and basically copy the last part but call it something new and point directly to your GD.

[gd-crypt]
type = crypt
remote = gd-cache:/encrypted
filename_encryption = standard
directory_name_encryption = true
password =Very long

to

[gdtest]
type = crypt
remote = gdrive:/encrypted
filename_encryption = standard
directory_name_encryption = true
password =Very long

You should be able to test with a rclone ls gdtest: --fast-list

and let it run to list out everything.

If you aren't see everything there, it should give you some more info.

Added that into my rclone config. Rebooted and the rclone ls command shows nothing. Is it because my API key is from a different gmail account from the one my drive is connected to?

Should not be. Did you run rclone config and get a new key authorized as well?

At the start, I used default but when I change to my own, I reauthorised and later on, started from scratch.

Just deleted my cache-temp-upload folder contents and did a cp -r from local (where current local media is stored) to my /mnt/PlexRemote folder (rclone mount folder.)

Anything copied to rclone mount folder should automatically appear while it's also in cache tmp folder being uploaded until it finishes uploading right?

If you removed your cache-tmp contents, you'd have to clear the cache database as well with a --cache-db-purge so it know it is gone as well.

If your goal is just to copy to get stuff there, you can use run rclone copy instead as that'll go much faster and you can copy that directly to the encrypted remote you setup.

The cache mount will pick it up in about ~1 minute.

If you copy to the mount, you are correct as you'd see it in the cache tmp folder first until the wait time expires and it uploads.

1 Like

Deleted my cache-db file and it seems to mount fine (after deleting everything from GD drive etc.)

Started an rclone copy to my encrypted mount gd-crypt:.

I tried to mount the folder while the copy process is running but I received this error.

2019/05/30 19:01:55 ERROR : /root/.cache/rclone/cache-backend/gd-cache.db: Error opening storage cache. Is there another rclone running on the same remote? failed to open a cache connection to "/root/.cache/rclone/cache-backend/gd-cache.db": timeout
2019/05/30 19:01:55 Failed to create file system for "gd-crypt:": failed to make remote gd-cache:"/encrypted" to wrap: failed to start cache db: failed to open a cache connection to "/root/.cache/rclone/cache-backend/gd-cache.db": timeout

Do I have to wait until rclone copy finishes to mount it?

Only one process can access the cache at the same time.

Does maxing your upload affect your download speed? After starting rclone (using my maximum upload 10mbps) copy, it halves my usual download speed for 40 mbps to 20 mbps.

In general maxing bandwidth in either direction can affect the other. Not a rclone thing just a general statement.