My VFS SweetSpot - Updated 30-Aug-2018


#543

What’s your mount command that you are using? What’s the version you are running?

What other services are you using along with it? Did you happen to just install Bazaar?


#544

Do you see these 2 graphs as well in the console for the last 12 hours or so?


#545

Hi ! Thanks for the prompt reply:

What’s your mount command that you are using?

usr/bin/rclone mount CryptSansCache: /media/GD
–allow-other
–fast-list
–buffer-size 256M
–dir-cache-time 72h
–drive-chunk-size 32M
–log-level DEBUG
–log-file /home/sorg/logs/rclone.log
–timeout 1h
–umask 002
–vfs-read-chunk-size 128M
–vfs-read-chunk-size-limit off
–rc

What’s the version you are running?

rclone v1.45-082-g781142a7-beta

  • os/arch: linux/amd64
  • go version: go1.11.4

What other services are you using along with it?

Plex
Radarr
Sonarr
ombi

Did you happen to just install Bazaar?

No.
I am quite happy with Plex + subzero.

Do you see these 2 graphs as well in the console for the last 12 hours or so?
Sure !



#546

I’m guessing your debug log is probably huge, but if you key in those times on the spikes, I would think the first spike after 12pm, is there something you can share from the logs?

I’m guessing there should be a lot of “O_RDONLY” lines along with “Flush|O_” items in there. I’d like to see if something caused a lot of opens/closes.

Was there a specific reason you moved to that beta version over running the stable?


#547

The debug log is not so huge. It was set on “INFO” until i noticed the issue 1 hour ago. I am now on DEBUG level.

I have nothing in the log between 09:52 an 19:00:
The error at 09:52 is another issue i have randomly. Some files looks corrupted. I have not investigated this problem yet.

2019/01/15 09:52:14 ERROR : Series/Versailles (2015)/S03/Versailles.(2015).S03E01.HDTV-1080p.hybris.x264_EAC3[FR+EN]_[FR].mkv: ReadFileHandle.Read error: low level retry 10/10: fail
ed to authenticate decrypted block - bad password?
2019/01/15 09:52:14 ERROR : Series/Versailles (2015)/S03/Versailles.(2015).S03E01.HDTV-1080p.hybris.x264_EAC3[FR+EN]_[FR].mkv: ReadFileHandle.Read error: failed to authenticate decr
ypted block - bad password?
2019/01/15 09:52:14 ERROR : Series/Versailles (2015)/S03/Versailles.(2015).S03E01.HDTV-1080p.hybris.x264_EAC3[FR+EN]_[FR].mkv: ReadFileHandle.Release error: file already closed
2019/01/15 19:00:46 ERROR : Films/Asterix and Cleopatra (1968)/Astérix.et.Cléopâtre.1968.WEBDL-1080p.Radarr.tt0062687.[FR].mkv: ReadFileHandle.Read error: low level retry 1/10: coul
dn't reopen file with offset and limit: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded
2019/01/15 19:00:46 ERROR : Films/Asterix and Cleopatra (1968)/Astérix.et.Cléopâtre.1968.WEBDL-1080p.Radarr.tt0062687.[FR].mkv: ReadFileHandle.Read error: low level retry 2/10: coul
dn't reopen file with offset and limit: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded

And below some parts of the logs since i have switched to DEBUG:

2019/01/15 21:36:06 NOTICE: Serving remote control on http://127.0.0.1:5572/
2019/01/15 21:40:41 DEBUG : rclone: Version "v1.45-082-g781142a7-beta" starting with parameters ["/usr/bin/rclone" "mount" "CryptSansCache:" "/media/GD" "--allow-other" "--fast-list
" "--buffer-size" "256M" "--dir-cache-time" "72h" "--drive-chunk-size" "32M" "--log-level" "DEBUG" "--log-file" "/home/sorg/logs/rclone.log" "--timeout" "1h" "--umask" "002" "--vfs-
read-chunk-size" "128M" "--vfs-read-chunk-size-limit" "off" "--rc"]
2019/01/15 21:40:41 NOTICE: Serving remote control on http://127.0.0.1:5572/
2019/01/15 21:40:41 DEBUG : Using config file from "/home/sorg/.rclone.conf"
2019/01/15 21:40:42 DEBUG : Encrypted drive 'CryptSansCache:': Mounting on "/media/GD"
2019/01/15 21:40:42 DEBUG : Adding path "vfs/forget" to remote control registry
2019/01/15 21:40:42 DEBUG : Adding path "vfs/refresh" to remote control registry
2019/01/15 21:40:42 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2019/01/15 21:40:42 DEBUG : : Root:
2019/01/15 21:40:42 DEBUG : : >Root: node=/, err=<nil>
2019/01/15 21:40:52 DEBUG : /: Lookup: name="DlTmp"
2019/01/15 21:40:52 DEBUG : /: >Lookup: node=DlTmp/, err=<nil>
2019/01/15 21:40:52 DEBUG : DlTmp/: Attr:
2019/01/15 21:40:52 DEBUG : DlTmp/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:40:52 DEBUG : DlTmp/: Lookup: name="watch"
2019/01/15 21:40:52 DEBUG : DlTmp/: >Lookup: node=<nil>, err=no such file or directory
2019/01/15 21:41:02 DEBUG : /: Lookup: name="DlTmp"
2019/01/15 21:41:02 DEBUG : /: >Lookup: node=DlTmp/, err=<nil>
2019/01/15 21:41:02 DEBUG : DlTmp/: Attr:
2019/01/15 21:41:02 DEBUG : DlTmp/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:02 DEBUG : /: Lookup: name="DlTmp"`
2019/01/15 21:41:02 DEBUG : DlTmp/: Attr:
2019/01/15 21:41:02 DEBUG : DlTmp/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:02 DEBUG : DlTmp/: Lookup: name="watch"
2019/01/15 21:41:02 DEBUG : DlTmp/: >Lookup: node=<nil>, err=no such file or directory
2019/01/15 21:41:05 DEBUG : /: Lookup: name="DlTmp"
2019/01/15 21:41:05 DEBUG : /: >Lookup: node=DlTmp/, err=<nil>
2019/01/15 21:41:05 DEBUG : DlTmp/: Attr:
2019/01/15 21:41:05 DEBUG : DlTmp/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:05 DEBUG : /: Lookup: name="DlTmp"
2019/01/15 21:41:05 DEBUG : /: >Lookup: node=DlTmp/, err=<nil>
2019/01/15 21:41:05 DEBUG : DlTmp/: Attr:
2019/01/15 21:41:05 DEBUG : DlTmp/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:05 DEBUG : DlTmp/: Lookup: name="completed"
2019/01/15 21:41:05 DEBUG : DlTmp/: >Lookup: node=DlTmp/completed/, err=<nil>
2019/01/15 21:41:05 DEBUG : DlTmp/completed/: Attr:
2019/01/15 21:41:05 DEBUG : DlTmp/completed/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:05 DEBUG : /: Attr:
2019/01/15 21:41:05 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:05 DEBUG : : Statfs:
2019/01/15 21:41:06 DEBUG : : >Statfs: stat={Blocks:274877906944 Bfree:272493705277 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<n
il>
2019/01/15 21:41:06 DEBUG : : Statfs:
2019/01/15 21:41:06 DEBUG : : >Statfs: stat={Blocks:274877906944 Bfree:272493705277 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<n
il>
2019/01/15 21:41:12 DEBUG : /: Lookup: name="DlTmp"
2019/01/15 21:41:12 DEBUG : /: >Lookup: node=DlTmp/, err=<nil>
2019/01/15 21:41:12 DEBUG : DlTmp/: Attr:
2019/01/15 21:41:12 DEBUG : DlTmp/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:12 DEBUG : /: Lookup: name="DlTmp"
2019/01/15 21:41:12 DEBUG : /: >Lookup: node=DlTmp/, err=<nil>
2019/01/15 21:41:12 DEBUG : DlTmp/: Attr:
2019/01/15 21:41:12 DEBUG : DlTmp/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2019/01/15 21:41:12 DEBUG : DlTmp/: Lookup: name="watch"
2019/01/15 21:41:12 DEBUG : DlTmp/: >Lookup: node=<nil>, err=no such file or directory
2019/01/15 21:41:22 DEBUG : /: Lookup: name="DlTmp"
2019/01/15 21:41:22 DEBUG : /: >Lookup: node=DlTmp/, err=<nil>
2019/01/15 21:41:22 DEBUG : DlTmp/: Attr:

I don’t see any “O_RDONLY” or “Flush|O_” items ?

I was running the beta long ago when the “cache” feature (that i don’t use anymore) was only available in beta. I can now revert to stable version.


#548

OK, I have the O_RDONLY flag.
Something is scanning my disk, because, i am not myself accessing those files.

cat /home/sorg/logs/rclone.log | grep O_RDONLY`
2019/01/15 21:50:25 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:25 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:26 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:26 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:27 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:27 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:28 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:28 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:28 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:29 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:29 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:30 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:31 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:31 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:31 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:32 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:32 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:50:33 DEBUG : DlTmp/Series/Le.Bureau.Des.Legendes.S04.EP02.mp4: Open: flags=O_RDONLY
2019/01/15 21:55:06 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E01.Pilot.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:06 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E02.And the Break-up Scene.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:07 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E03.And Strokes of Goodwill.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:07 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E04.And the Rich People Problems.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:08 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E05.And the '90s Horse Party.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:08 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E06.And the Disappearing Bed.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:09 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E07.And the Pretty Problem.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:09 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E08.And Hoarder Culture.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:10 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E09.And the Really Petty Cash.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:10 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E10.And the Very Christmas Thanksgiving.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:11 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E11.And the Reality Check.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:11 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E12.And the Pop-Up Sale.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:12 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E13.And the Secret Ingredient.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:13 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E14.And the Upstairs Neighbor.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:13 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E15.And the Blind Spot.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:14 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E16.And the Broken Hearts.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:14 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E17.And the Kosher Cupcakes.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:15 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E18.And the One-Night Stands.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:15 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E19.And the Spring Break.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:16 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E20.And the Drug Money.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:16 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E21.And the Messy Purse Smackdown.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:17 DEBUG : Series/2 Broke Girls/S01/2 Broke Girls.S01E22.And the Big Buttercream Breakthrough.SD TV.SICKBEARD.nfo: Open: flags=O_RDONLY
2019/01/15 21:55:17 DEBUG : Series/2 Broke Girls/S02/2 Broke Girls.S02E01.And the Hidden Stash.HD TV.SICKBEARD.nfo: Open: flags=O_RDONLY

But no FlushIO


#549

Sorry as that didn’t quite paste the way I wanted.

If you see the opens, you should see a close or a flush after.

2019/01/15 16:49:41 DEBUG : mounted: Open: flags=OpenReadOnly
2019/01/15 16:49:41 DEBUG : mounted: Open: flags=O_RDONLY
2019/01/15 16:49:41 DEBUG : mounted: >Open: fd=mounted (r), err=<nil>
2019/01/15 16:49:41 DEBUG : mounted: >Open: fh=&{mounted (r)}, err=<nil>
2019/01/15 16:49:41 DEBUG : &{mounted (r)}: Flush:
2019/01/15 16:49:41 DEBUG : &{mounted (r)}: >Flush: err=<nil>
2019/01/15 16:49:41 DEBUG : &{mounted (r)}: Release:
2019/01/15 16:49:41 DEBUG : &{mounted (r)}: >Release: err=<nil>

I think “Flush:” would be enough to look at or maybe the Release:.

I’d want to figure out what is causing the multiple opens/closes on the files.

You might be able to use lsof to see. I have a mergerfs in the middle so I have to backtrack through and see what’s going on.

felix@gemini:~$ sudo lsof /GD
COMMAND   PID USER   FD   TYPE DEVICE   SIZE/OFF                 NODE NAME
mergerfs 3378 root  119r   REG   0,41 5867157863 17717103958411265932 /GD/Radarr_Movies/Smallfoot (2018)/Smallfoot (2018).mkv
felix@gemini:~$ sudo lsof /gmedia | grep -v torrent
COMMAND    PID  USER   FD   TYPE DEVICE    SIZE/OFF   NODE NAME
Plex\x20T 1873 felix    6r   REG   0,44  5867157863 255276 /gmedia/Radarr_Movies/Smallfoot (2018)/Smallfoot (2018).mkv

As an example, I can tell Plex is using the file.


#550

I can see 1 film being read by plex (it is on the local part of the mergefs, so no problem) and radarr scanning my library.

sudo lsof /media/gmedia | grep -v transmiss
COMMAND     PID USER   FD   TYPE DEVICE    SIZE/OFF  NODE NAME
Plex       2512 root    5r   REG   0,36  1060220199 36909 /media/GSuiteCrypt/Films/Hunter Killer (2018)/Hunter.Killer.2018.WEBDL-720p.NoTag.tt1846589.[FR].mkv
mono      10553 root   20r   REG   0,36    89643999 54815 /movies/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD-trailer.mov

In the logs, i can see the “Release” but i think there is nothing to analyze currently as it is curently retrying in loop to access the files… What would be interesting is the log before the ban, but i don’t have it.

2019/01/15 23:12:15 DEBUG : &{Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD-trailer.mov (r)}: Release:
2019/01/15 23:12:15 DEBUG : &{Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD-trailer.mov (r)}: >Release: err=<nil>
2019/01/15 23:12:15 DEBUG : Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso: Open: flags=OpenReadOnly
2019/01/15 23:12:15 DEBUG : Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso: Open: flags=O_RDONLY
2019/01/15 23:12:15 DEBUG : Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso: >Open: fd=Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso (r), err=<nil>
2019/01/15 23:12:15 DEBUG : Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso: >Open: fh=&{Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso (r)}, err=<nil>
2019/01/15 23:12:16 DEBUG : &{Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso (r)}: Release:
2019/01/15 23:12:16 DEBUG : &{Films/To Rome with Love (2012)/To Rome with Love (2012).tt1859650.SD.iso (r)}: >Release: err=<nil>
2019/01/15 23:12:26 DEBUG : Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov: Open: flags=OpenReadOnly
2019/01/15 23:12:26 DEBUG : Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov: Open: flags=O_RDONLY
2019/01/15 23:12:26 DEBUG : Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov: >Open: fd=Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov (r), err=<nil>
2019/01/15 23:12:26 DEBUG : Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov: >Open: fh=&{Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov (r)}, err=<nil>
2019/01/15 23:12:41 DEBUG : &{Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov (r)}: Release:
2019/01/15 23:12:41 DEBUG : &{Films/Transcendence (2014)/Transcendence (2014).tt2209764.HD-trailer.mov (r)}: >Release: err=<nil>
2019/01/15 23:12:44 DEBUG : Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov: Open: flags=OpenReadOnly
2019/01/15 23:12:44 DEBUG : Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov: Open: flags=O_RDONLY
2019/01/15 23:12:44 DEBUG : Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov: >Open: fd=Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov (r), err=<nil>
2019/01/15 23:12:44 DEBUG : Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov: >Open: fh=&{Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov (r)}, err=<nil>
2019/01/15 23:13:16 DEBUG : &{Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov (r)}: Release:
2019/01/15 23:13:16 DEBUG : &{Films/Transformers Age of Extinction (2014)/Transformers Age of Extinction (2014).tt2109248.HD-trailer.mov (r)}: >Release: err=<nil>
2019/01/15 23:13:49 DEBUG : Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi: Open: flags=OpenReadOnly
2019/01/15 23:13:49 DEBUG : Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi: Open: flags=O_RDONLY
2019/01/15 23:13:49 DEBUG : Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi: >Open: fd=Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi (r), err=<nil>
2019/01/15 23:13:49 DEBUG : Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi: >Open: fh=&{Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi (r)}, err=<nil>
2019/01/15 23:13:50 DEBUG : &{Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi (r)}: Release:
2019/01/15 23:13:50 DEBUG : &{Films/X-Men Days of Future Past (2014)/X-Men.Days.of.Future.Past.2014.SDTV.Men.tt1877832.avi (r)}: >Release: err=<nil>
2019/01/15 23:13:55 DEBUG : Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4: Open: flags=OpenReadOnly
2019/01/15 23:13:55 DEBUG : Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4: Open: flags=O_RDONLY
2019/01/15 23:13:55 DEBUG : Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4: >Open: fd=Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4 (r), err=<nil>
2019/01/15 23:13:55 DEBUG : Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4: >Open: fh=&{Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4 (r)}, err=<nil>
2019/01/15 23:14:01 DEBUG : &{Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4 (r)}: Release:
2019/01/15 23:14:01 DEBUG : &{Films/Yves Saint Laurent (2014)/Yves Saint Laurent (2014).tt2707858.HD-trailer.mp4 (r)}: >Release: err=<nil>
2019/01/15 23:14:06 DEBUG : Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov: Open: flags=OpenReadOnly
2019/01/15 23:14:06 DEBUG : Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov: Open: flags=O_RDONLY
2019/01/15 23:14:06 DEBUG : Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov: >Open: fd=Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov (r), err=<nil>
2019/01/15 23:14:06 DEBUG : Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov: >Open: fh=&{Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov (r)}, err=<nil>
2019/01/15 23:14:13 DEBUG : &{Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov (r)}: Release:
2019/01/15 23:14:13 DEBUG : &{Films/The Zero Theorem (2013)/The Zero Theorem (2013).tt2333804.HD-trailer.mov (r)}: >Release: err=<nil>

#551

I don’t think you’ll be able to figure out the cause now as you have the ban going on.

It would have been in the debug logs to figure out what file got opened/closed/repeatedly as that to my knowledge, is what caused the download quota on the file.

So in the example, something open/closes the file hundreds/thousands of times to generate the error. Without capturing the log, I don’t think you’d be able to figure it out.

It could also be a rclone defect/bug, but without the logs, it really is just guessing.

If it was me, I’d leave it in DEBUG and see if it happens again, but you have to wait the ~24 hours for the download quota limit to go away.


#552

This what i am doing. I’ll follow up at the end of the ban.

I am also thinking i have not spoken of Transmission:
I am downloading to a directory kept always on the local part of the mergefs.
But once complete, the file is moved (the next night) to the GDrive mount.
It may be the seeding of this files that cause many access to the drive (although i am not sure as I don’t see so many access to the mount on the graph).


#553

You would need to give a test with debug log starting and using transmission. I don’t torrent at all from my mount as I have local storage for that since a cloud mount would be a very poor case for cloud storage in the way that it works.


#554

How big is your plex data for 150TB+ movies?


#555

I have SubZero getting the subtitles for my shows in plex, how can I make sure they are also uploaded to gdrive ? I want to keep them there in the appropriate remote folders aswell.


#556

I just use my regular setup with SubZero and let everything happen automatically like a movie or anything else.


#557

So just to be sure I understand correctly. When I start a movie which is on my remote, then Subzero will create an srt file on the remote aswell ?
Or am I getting that completly wrong ?
I have the setup almost the same as you i think. I upload my stuff at night. But only once it is on gdrive plex will scan it for the first time. Before that it is not in plex.


#558

It depends on how you have Subzero configured. I get my subs before it plays and the subs along with the movies are uploaded at night as well.


#559

Could you maybe share the part of the subzero config where I can set that. I want to have it like you have it configured.


#560

I do forced subtitles only.

and I keep them with the media for me.


#561

OK.

I have been banned again yesterday around 13:15:52 french time.

Here is an exctract of the logs before it happens. Apparently some scan was happening.

PS: browsing a such huge logfile is a nightmare. Any idea if there is a good tool for that ?


#562

So what’s your mount command / rclone version / looks like you have optimization going on.

That one file was opened 42 times in the span of 30 seconds in your log. Are you using any other plugins / Bazaar ?

Is deep analysis turned on?