Rclone and emby settings to avoid pulling entire files on scan

What is the problem you are having with rclone?

I'm trying to figure out what settings to use so entire files aren't pulled down by Emby. I've disabled thumbnail generation and subtitle pulls. Something is causing the files to be pulled though. I checked the cache, they aren't partials, they are the entire files, completely playable.

I did read that real time monitoring doesn't work over an rclone mount, that's fine, I'll turn that off, I don't imagine it would cause the files to be pulled.

I did notice in Emyb's log, it's trying to do an ffprobe, the line looks to me like it should just read the metadata header, but it timed out on each of the 6 files, which seems to me like it was waiting for rclone to pull the whole file down before being made accessible, so that doesn't seem like what I'm going for, but I still am not sure who the culprit is here.

Thank you!

What is your rclone version (output from rclone version)

rclone v1.52.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Win10 x64

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

 rclone mount secV:/ V: --fuse-flag --VolumePrefix=\server\GDriveSecV --dir-cache-time=1000h --poll-interval 15s --cache-dir="Z:\rclone.cache" --vfs-cache-mode full --vfs-cache-max-size 50G --vfs-cache-max-age 12h --bwlimit 20M -v

The rclone config contents with secrets removed.

[gd]
type = drive
client_id = xxxx.apps.googleusercontent.com
client_secret = xxxx
scope = drive
root_folder_id = xxxx
token = {"access_token":"xxxx"}

[secV]
type = crypt
remote = gd:/secV
filename_encryption = standard
directory_name_encryption = true
password = xxxx


A log from the command with the -vv flag

this log probably isn't useful, I had to restart with -vv, I normally only run -v; and it doesn't cover a time period when I added a new file to be scanned. I'll work on capturing that.

2020/10/08 15:12:53 DEBUG : rclone: Version "v1.52.3" starting with parameters ["C:\\rclone\\rclone.exe" "mount" "secV:/" "V:" "--fuse-flag" "--VolumePrefix=\\server\\GDriveSecV" "--dir-cache-time=1000h" "--poll-interval" "15s" "--cache-dir=Z:\\rclone.cache" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "50G" "--vfs-cache-max-age" "12h" "--bwlimit" "20M" "-vv"]
2020/10/08 15:12:53 DEBUG : Using config file from "C:\\Users\\emby\\.config\\rclone\\rclone.conf"
2020/10/08 15:12:53 INFO  : Starting bandwidth limiter at 20MBytes/s
2020/10/08 15:12:54 DEBUG : Encrypted drive 'secV:/': Mounting on "V:"
2020/10/08 15:12:54 DEBUG : vfs cache root is "Z:\\rclone.cache\\vfs\\secV"
2020/10/08 15:12:54 DEBUG : fs cache: renaming cache item "Z:\\rclone.cache\\vfs\\secV" to be canonical "//?/Z:/rclone.cache/vfs/secV"
2020/10/08 15:12:54 DEBUG : Adding path "vfs/forget" to remote control registry
2020/10/08 15:12:54 DEBUG : Adding path "vfs/refresh" to remote control registry
2020/10/08 15:12:54 DEBUG : TV.Concluded/Ad Vitam/Season 1/Ad Vitam - S01E02 - Episode 2 WEBDL-1080p.mkv: updateTime: setting atime to 2020-10-08 14:12:19.8327043 -0400 EDT
2020/10/08 15:12:54 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2020/10/08 15:12:54 DEBUG : Encrypted drive 'secV:/': Mounting with options: ["-o" "fsname=secV:/" "-o" "subtype=rclone" "-o" "max_readahead=131072" "-o" "attr_timeout=1" "-o" "atomic_o_trunc" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=secV" "--VolumePrefix=\\server\\GDriveSecV"]
2020/10/08 15:12:54 DEBUG : TV.GDTest/Ad Vitam/Season 1/Ad Vitam - S01E01 - Episode 1 WEBDL-1080p.mkv: updateTime: setting atime to 2020-10-08 15:05:12.1346206 -0400 EDT
2020/10/08 15:12:54 DEBUG : TV.GDTest/Ad Vitam/Season 1/Ad Vitam - S01E02 - Episode 2 WEBDL-1080p.mkv: updateTime: setting atime to 2020-10-08 14:57:48.3900201 -0400 EDT
2020/10/08 15:12:54 DEBUG : TV.GDTest/Ad Vitam/Season 1/Ad Vitam - S01E03 - Episode 3 WEBDL-1080p.mkv: updateTime: setting atime to 2020-10-08 14:57:48.4099665 -0400 EDT
2020/10/08 15:12:54 DEBUG : TV.GDTest/Ad Vitam/Season 1/Ad Vitam - S01E04 - Episode 4 WEBDL-1080p.mkv: updateTime: setting atime to 2020-10-08 14:57:54.0065737 -0400 EDT
2020/10/08 15:12:54 DEBUG : TV.GDTest/Ad Vitam/Season 1/Ad Vitam - S01E05 - Episode 5 WEBDL-1080p.mkv: updateTime: setting atime to 2020-10-08 14:57:48.3989968 -0400 EDT
2020/10/08 15:12:54 DEBUG : TV.GDTest/Ad Vitam/Season 1/Ad Vitam - S01E06 - Episode 6 WEBDL-1080p.mkv: updateTime: setting atime to 2020-10-08 14:57:48.3930124 -0400 EDT
2020/10/08 15:12:54 DEBUG : TV.GDTest/Ad Vitam/poster.jpg: updateTime: setting atime to 2020-10-08 14:48:53.7968362 -0400 EDT
2020/10/08 15:12:54 INFO  : Cleaned the cache: objects 15 (was 0), total size 13.950G (was 0)
2020/10/08 15:12:54 DEBUG : Encrypted drive 'secV:/': Init:
2020/10/08 15:12:54 DEBUG : Encrypted drive 'secV:/': >Init:
2020/10/08 15:12:54 DEBUG : /: Statfs:
2020/10/08 15:12:55 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:277123071711 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Readlink:
2020/10/08 15:12:55 DEBUG : /: >Readlink: linkPath="", errc=-40
The service rclone has been started.
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /Desktop.ini: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:12:55 DEBUG : /Desktop.ini: >Getattr: errc=-2
2020/10/08 15:12:55 DEBUG : /Desktop.ini: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /Desktop.ini: >Getattr: errc=-2
2020/10/08 15:12:55 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /AutoRun.inf: >Getattr: errc=-2
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2020/10/08 15:12:55 DEBUG : /: >Getattr: errc=0
2020/10/08 15:12:55 DEBUG : /: Opendir:
2020/10/08 15:12:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2020/10/08 15:12:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2020/10/08 15:12:55 DEBUG : /: >Opendir: errc=0, fh=0x0
2020/10/08 15:12:55 DEBUG : /: Releasedir: fh=0x0
2020/10/08 15:12:55 DEBUG : /: >Releasedir: errc=0
2020/10/08 15:13:09 DEBUG : Google drive root 'secV': Checking for changes on remote


You will need to upgrade rclone to the latest stable i.e. 1.53.2. Prior to 1.53 setting vfs-cache-mode to full will download the whole file instead of only the requested chunks.

Ok! I'll give that a try! Thanks!

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.