New rclone cache feature - files randomly disappear

#1

Hello,
I’m using latest rclone-beta cache feature with my dockerized plex media server this way:

Google Drive -> Cache -> Crypt

rclone v1.38-200-ga1f8318bβ
- os/arch: linux/amd64
- go version: go1.9.2

rclone.conf

[gdrive]
type = drive
client_id = XXXXX
client_secret = XXXXX
token = XXXXX

[encgdrive]
type = crypt
remote = gdcache:/folder
filename_encryption = standard
password = XXXXX
password2 =

[gdcache]
type = cache
remote = gdrive:
chunk_size = 40M
info_age = 1h
chunk_age = 4h
warmup_age = 6h

sudo /usr/local/bin/rclone-beta mount --max-read-ahead=512m --gid 1000 --uid 1000 --read-only --allow-non-empty --allow-other -vv encgdrive:/ /mnt/plexd

Library scan works well right after mounting, but after time, some movies suddenly disappear randomly.
Taking a look into the folders of the disappeared movies, all folders and subfolders are still there, only the movie-files (.mp4) are gone.
If I remount with --cache-db-purge, everything works fine again… for a few hours, then the movies start to disappear randomly again.

Any advice? Thanks in advance

#2

I have issues with files and folder which goes missing from the cache but still is present on the cloud https://github.com/ncw/rclone/issues/1876

#4

It seems like it has something to do with info_age.
If I set info_age = 15m, half of the movies are gone about 15 minutes after library scan.

New feature: CACHE
#5

I am seeing this exact issue too, example:

rclone ls cde:tv/Brooklyn.Nine-Nine.S02.1080p.WEB-DL.x264 --cache-db-purge
will list the entire contents of the folder (23 files) however the mount of this folder only shows the first file. This is occurring on several different folders.

Any ideas?

#6

I have been running into this behavior on a variety of folders since using the rclone cache mount vs. plexdrive.

Using the following:

rclone v1.39

  • os/arch: linux/amd64
  • go version: go1.9.2

Here’s what I know about this behavior (so far):

  1. Have only ever observed this when mounting a remote that is a decrypt of an existing cache remote in rclone
  2. Using rclone ls against the SAME REMOTE that I’m using to setup my mount, I see all files in the folder listed as expected
  3. Using ls against the mount, I get some sub-set of the files that exist in the folder…there does not appear to be a pattern to which files show and which do not.
  4. Performing a server-side rename of the folder (using the “moveto” command completely outside the context of any rclone cache mount) produces some interesting (maybe helpful?) results:
    a. Rebuilding the cache usually now picks up all the files that were missing before in the NEW directory
    b. The OLD DIRECTORY that I tried to rename still sticks around
    c. The files that showed up in the OLD DIRECTORY (when using ls against the mount - point 3 above) are the only files left - it’s as if they did not move with the server-side move
    d. All files show in the newly created directory (even the ones that “refused” to move - they now exist in both places)

Hope that helps!

#7

I’d suggest trying with 170 or newer 1.39 beta, solved a few issues similar to this for me.

#8

Thanks, I updated to

rclone v1.39-171-g1383df4fβ

  • os/arch: linux/amd64
  • go version: go1.10

But I still see the same behavior. The same files are missing from the same folders when looking at the decrypted rclone cache mount. Plexdrive sees them fine, rclone sees them fine with an “ls” against the (non-cached) remote.

Happy to provide more info if it will help…this seems like a tricky issue to find.

#9

Just a thought but did you purge the cache on mount?

#10

Yes. It’s one of my mount options. There’s just something different about those files that I cannot put my finger on.

#11

Same here, although it seems to be much more predictible in my case.

I’ve created a new cache remote for my existing Gdrive remote that is wrapped into a crypting remote, so the order is GDrive -> GDCache -> GDCrypt. With nothing currently mounted, i can run the usual commands like ‘ls’ and ‘lsd’ which would deliver expected results.

but after 5 minutes (the interval seems to be pretty exact as i’ve done it three times now) all files and folders disappear except one single file that is located in the root folder of my GDcrypt remote (all the other stuff is nested in sub-folders).

this status remains until a --cache-db-purge command is sent with ac ls or lsd command to the GDcrypt remote which brings back all the content for another five minutes.

The GDcache remote shows the same contents every single time. It shows a bunch of encrypted directories and files as expected.

runnint the GDcrypt mount with -vv, tons of debugging info shows up but no errors.

a Re-Read takes place after about 5m and after this action, there is only one item remaining.

Initially every item of my GDcrypt remote is listed in the way like

2018/02/23 12:46:52 DEBUG : /Folder_XYZ: Getattr: fh=0xFFFFFFFFFFFFFFFF
2018/02/23 12:46:52 DEBUG : /Folder_XYZ: >Getattr: errc=0

When the Re-Read is happening, only the one single file located in the root directory becomes listed as a result for the Re-Read.

Files disappearing after upload
#12

@ncw tagging you in just in case this got lost in the shuffle.

Is there an existing ticket open for this, if not, what info do you need that is missing?

#13

If you can make a recipe for someone to reproduce this with the latest beta then it is definitely worth making a new issue on github

#14

I’ve been experiencing a very similar issue with all versions since the OP had and am still getting it with v1.42 - hoping someone else has experienced this as well?

Similar situation to the above, a gdrive-cache-crypt mount with Plex (which gets 'rclone synce’d elsewhere) is randomly ‘losing’ a bunch of directories whilst they are present in a ‘rclone lsd’ - doing a SIGHUP or rc/expire magically brings them back.

I am however struggling to replicate it in a predictable manner to have such a recipe to raise a bugreport with.

Hoping someone else has observed this with possible trigger-points.

EDIT: Thanks to others - it appears this is related to bug #2354