What’s your rclone version?
felix@gemini:~$ rclone version
rclone v1.45
- os/arch: linux/amd64
- go version: go1.11.2
RClone version is:
rclone v1.45
- os/arch: linux/amd64
- go version: go1.11.2Did the ban happen before from plexdrive and it’s still going?
I haven’t seen vfs create a ban personally as it does chunked reading so that issue was fixed a couple versions back.
Did you try running and older version first and upgrade recently which may have caused it?
Not much to do other than wait it out and retry unfortunately.
I’ll give it a shot tonight - seems I’m banned for approx 14 hours each time.
No I turned off Plexdrive days ago. Rclone VFS worked last night, was able to stream content etc did a scan of my Movies folder.
Did a scan of my TV folder this morning and bam!
My only thought is when scanning TV folders, there’s quite a lot to process (/TV/TV Show Name/Season/episode.ext) and maybe that is too much for the config?
I’m not running a separate cache. I’m also not running any rc commands. Do you think that could be an issue?
(Thanks for your help BTW)
Are you using your own API key as well?
I’ve scanned 30-40TB over a few days with no issues with Plex and repeated with Emby during the same period with no issues ever.
The errors you are seeing above were there in earlier versions as vfs didn’t have a chunked reading method so each read of a file looked like a full download and Plex opens and closes the file alot when scanning.
I don’t use cache as the rc comand is only there for refreshing the directory list and wouldn’t matter much as that’s more for ease of use than anything else.
If you are sure the ban is gone, please run it in debug log level so it’ll capture the issue if there is something going on. It generates a lot of logs, but only way to really see what is going on unfortunately.
Yeah very strange. I’ve been reading how no-one else is having any issues.
Definitely using v1.45 of Rclone
As for my own API key - no-one else is using it if thats what you mean?
Yeah, just meaning you create your own API key to use for rclone rather than using the rclone one. That would not create download quota errors though as that helps with the rate limiting since it’s shared.
Ok yes, using my own API key in that regard.
Running a library scan at 9pm seemed to be ok, running it at 5am the next morning triggered a ban
I’ve shut down Radarr/Sonarr (which is running on a different server), and turned off all Plex library scans and scheduled tasks related to scanning - so I’m at a loss as to what is triggering the ban every morning.
I’ll change logging to debug and run a scan once the ban opens again.
Animosity022,
I wanted to say thank you. I’ve been lurking this thread and have copied your mergerfs set-up.
I found it difficult to conceptualize at first, but would you say this diagram describes the mergerfs/rclone set-up?
Yep, I think that’s spot on. Any writes go local.
Any reads are from the cloud, unless they are local.
Local files are moved up nightly for me.
Are you able to write directly to the gdrive mount with VFS? Or is there an advantage to caching it locally first?
Ok so back online after the ban, after 14 hours of absolutely no usage on the mount.
Remounted rclone and this is what the logs are showing so far:
018/11/29 19:34:38 DEBUG : rclone: Version "v1.45" starting with parameters ["/usr/bin/rclone" "mount" "plexcrypt:" "/home/media/plex" "--config" "/root/.config/rclone/rclone.conf" "--allow-other" "--allow-non-empty" "--dir-cache-time" "72h" "--drive-chunk-size" "32M" "--buffer-size" "1G" "--log-level" "DEBUG" "--log-file" "/home/media/logs/rclone.log" "--vfs-read-chunk-size" "128M" "--vfs-read-chunk-size-limit" "off"]
2018/11/29 19:34:38 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2018/11/29 19:34:40 DEBUG : Encrypted drive 'plexcrypt:': Mounting on "/home/media/plex"
2018/11/29 19:34:40 DEBUG : Adding path "vfs/forget" to remote control registry
2018/11/29 19:34:40 DEBUG : Adding path "vfs/refresh" to remote control registry
2018/11/29 19:34:40 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2018/11/29 19:34:40 DEBUG : : Root:
2018/11/29 19:34:40 DEBUG : : >Root: node=/, err=<nil>
2018/11/29 19:34:48 DEBUG : /: Attr:
2018/11/29 19:34:48 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:34:48 DEBUG : /: ReadDirAll:
2018/11/29 19:34:49 DEBUG : /: >ReadDirAll: item=6, err=<nil>
2018/11/29 19:34:49 DEBUG : /: Lookup: name="Movies"
2018/11/29 19:34:49 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2018/11/29 19:34:49 DEBUG : Movies/: Attr:
2018/11/29 19:34:49 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:34:49 DEBUG : /: Lookup: name="Music"
2018/11/29 19:34:49 DEBUG : /: >Lookup: node=Music/, err=<nil>
2018/11/29 19:34:49 DEBUG : Music/: Attr:
2018/11/29 19:34:49 DEBUG : Music/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:34:49 DEBUG : /: Lookup: name="Sport"
2018/11/29 19:34:49 DEBUG : /: >Lookup: node=Sport/, err=<nil>
2018/11/29 19:34:49 DEBUG : Sport/: Attr:
2018/11/29 19:34:49 DEBUG : Sport/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:34:49 DEBUG : /: Lookup: name="TV"
2018/11/29 19:34:49 DEBUG : /: >Lookup: node=TV/, err=<nil>
2018/11/29 19:34:49 DEBUG : TV/: Attr:
2018/11/29 19:34:49 DEBUG : TV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:35:40 DEBUG : Google drive root 'crypt': Checking for changes on remote
2018/11/29 19:35:43 DEBUG : /: Lookup: name="Movies"
2018/11/29 19:35:43 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2018/11/29 19:35:43 DEBUG : Movies/: Attr:
2018/11/29 19:35:43 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:35:43 DEBUG : /: Lookup: name="Movies"
2018/11/29 19:35:43 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2018/11/29 19:35:43 DEBUG : Movies/: Attr:
2018/11/29 19:35:43 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:35:43 DEBUG : Movies/: ReadDirAll:
2018/11/29 19:36:14 DEBUG : Movies/: >ReadDirAll: item=5645, err=<nil>
2018/11/29 19:36:14 DEBUG : /: Lookup: name="Movies"
2018/11/29 19:36:14 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/: Attr:
2018/11/29 19:36:14 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:14 DEBUG : /: Lookup: name="Movies"
2018/11/29 19:36:14 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/: Attr:
2018/11/29 19:36:14 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/: ReadDirAll:
2018/11/29 19:36:14 DEBUG : Movies/: >ReadDirAll: item=5645, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/: Lookup: name="#Stuck (2014)"
2018/11/29 19:36:14 DEBUG : Movies/: >Lookup: node=Movies/#Stuck (2014)/, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/#Stuck (2014)/: Attr:
2018/11/29 19:36:14 DEBUG : Movies/#Stuck (2014)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/: Lookup: name="#realityhigh (2017)"
2018/11/29 19:36:14 DEBUG : Movies/: >Lookup: node=Movies/#realityhigh (2017)/, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/#realityhigh (2017)/: Attr:
2018/11/29 19:36:14 DEBUG : Movies/#realityhigh (2017)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/: Lookup: name="'71 (2014)"
2018/11/29 19:36:14 DEBUG : Movies/: >Lookup: node=Movies/'71 (2014)/, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/'71 (2014)/: Attr:
2018/11/29 19:36:14 DEBUG : Movies/'71 (2014)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/: Lookup: name="'Til Death Do Us Part (2017)"
2018/11/29 19:36:14 DEBUG : Movies/: >Lookup: node=Movies/'Til Death Do Us Part (2017)/, err=<nil>
2018/11/29 19:36:14 DEBUG : Movies/'Til Death Do Us Part (2017)/: Attr:
2018/11/29 19:36:14 DEBUG : Movies/'Til Death Do Us Part (2017)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
And then this…
2018/11/29 19:36:17 DEBUG : Movies/: ReadDirAll:
2018/11/29 19:36:17 DEBUG : Movies/: >ReadDirAll: item=5645, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/: ReadDirAll:
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/: >ReadDirAll: item=1, err=<nil>
2018/11/29 19:36:17 DEBUG : /: Lookup: name="Movies"
2018/11/29 19:36:17 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/: Attr:
2018/11/29 19:36:17 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:17 DEBUG : /: Lookup: name="Movies"
2018/11/29 19:36:17 DEBUG : /: >Lookup: node=Movies/, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/: Attr:
2018/11/29 19:36:17 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/: Lookup: name="The Sweetest Heart (2018)"
2018/11/29 19:36:17 DEBUG : Movies/: >Lookup: node=Movies/The Sweetest Heart (2018)/, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/: Attr:
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/: Lookup: name="The Sweetest Heart (2018).mkv"
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/: >Lookup: node=Movies/The Sweetest Heart (2018)/The Sweetest Heart (2018).mkv, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/The Sweetest Heart (2018).mkv: Attr:
2018/11/29 19:36:17 DEBUG : Movies/The Sweetest Heart (2018)/The Sweetest Heart (2018).mkv: >Attr: a=valid=1s ino=0 size=4592598369 mode=-rw-r--r--, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/: ReadDirAll:
2018/11/29 19:36:17 DEBUG : Movies/: >ReadDirAll: item=5645, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/: Lookup: name="The Predator (2018)"
2018/11/29 19:36:17 DEBUG : Movies/: >Lookup: node=Movies/The Predator (2018)/, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/The Predator (2018)/: Attr:
2018/11/29 19:36:17 DEBUG : Movies/The Predator (2018)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:17 DEBUG : Movies/The Predator (2018)/: ReadDirAll:
2018/11/29 19:36:18 DEBUG : Movies/The Predator (2018)/: >ReadDirAll: item=1, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/The Predator (2018)/: Lookup: name="The Predator (2018).mkv"
2018/11/29 19:36:18 DEBUG : Movies/The Predator (2018)/: >Lookup: node=Movies/The Predator (2018)/The Predator (2018).mkv, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/The Predator (2018)/The Predator (2018).mkv: Attr:
2018/11/29 19:36:18 DEBUG : Movies/The Predator (2018)/The Predator (2018).mkv: >Attr: a=valid=1s ino=0 size=3986978873 mode=-rw-r--r--, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/: Attr:
2018/11/29 19:36:18 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/: ReadDirAll:
2018/11/29 19:36:18 DEBUG : Movies/: >ReadDirAll: item=5645, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/: Lookup: name="The Nun (2018)"
2018/11/29 19:36:18 DEBUG : Movies/: >Lookup: node=Movies/The Nun (2018)/, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/: Attr:
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/: ReadDirAll:
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/: >ReadDirAll: item=1, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/: Lookup: name="The Nun (2018).mkv"
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/: >Lookup: node=Movies/The Nun (2018)/The Nun (2018).mkv, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/The Nun (2018).mkv: Attr:
2018/11/29 19:36:18 DEBUG : Movies/The Nun (2018)/The Nun (2018).mkv: >Attr: a=valid=1s ino=0 size=8212462258 mode=-rw-r--r--, err=<nil>
2018/11/29 19:36:18 DEBUG : Movies/: Attr:
2018/11/29 19:36:18 DEBUG : Movies/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
Which all looks fine.
Drive usage looked high to begin with, and I started panicking, but realised my second server was also scanning (same config as this- so I stopped it).
Seem to pull around 200 API queries per Plex Library Scan with your config
All seems ok to me?
The trick will be whether the API dies again at 5am, like it has all week this week. I don’t seem to have any issues in the evening, but 5am comes around…
I’ve disabled everything I can think of. Sonarr/Radarr plus all the Plex Library Scan and analysis tasks. Also uninstalled the Sub-Zero plugin.
Wish me luck!
You can write directly to the mount but I find it better to use my setup as it allows me to use hardlinks locally to copy files from Radarr/Sonarr and upload based on my timing, which I do in a batch over night.
You mentioned you have 2 servers, does the other server have a different rclone version? Even with high API hits, you normally just get a rate limit warning and rclone backs off.
The download quota error is much more related to an older rclone version.
Nah I checked and they both are running 1.45.
Although it seems on the server I use for Sonarr/Radarr etc (another yes I know…), my main user is running 1.45, but root is saying 1.40.
There are no active configs under root though.
The 403 download quotas definitely come from earlier versions though, which would be the 1.40.
I’ve scanned repeatedly 40-50TB in a few days with Plex/Emby and never have seen a download quota error. I can instantly reproduce that error with an earlier version scanning a few files.
I’d be sure 
Great thanks - I’ve updated the Root Rclone to 1.45 just in case.
So a simple upgrade will suffice? Or will I need to add VFS to the config?
I’ve also tried using Sonarr/Radarr directly with VFS, but I kept getting Import errors (files missing) when the download client finished unpacking the the file (hardlinking issue for sure).
Going the mergerfs route, everything is done on a localdisk first and then rclone-copy to the cloud afterwards.
Only a few days now, but it’s soo much more stable than writing directly to VFS.
The defaults are fine and really don’t need to be touched. I keep them in for my own benefit.
There is a bit of work needed for writing directly and it doesn’t perform perfect as you need to use vfs-cache-mode writes and I find that to be a but clunky as it doesn’t delay uploads and starts the uploads immediately. It tends to hand Sonarr/Radarr while it copies as well so wasn’t an ideal solution for me. To make things snappier and available, I use mergerfs and my local setup as that meets my use case much better albeit it does add 1 layer of complexity which I normally don’t like.