Stream from rclone mounted google drive on Windows, player freezes every now and then

ldz · August 28, 2019, 2:59pm

What is the problem you are having with rclone?

I tried to stream videos from a rclone mounted Google drive disc on Windows, the media player will freeze every now and then.
I check the network activity at the time of freezing, it drops to 0kbs.
When I pause the video for a while and hit play button in a few minutes, the video can keep playing.
I also observed the network activity rise up before I hit the play button.

What is your rclone version (output from `rclone version`)

rclone v1.48.0

os/arch: windows/amd64
go version: go1.12.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 64bit 1903

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

set HTTP_PROXY=http://127.0.0.1:1080

rclone mount video:/ L: --log-file log.log --buffer-size 512M --low-level-retries 200 --allow-other --dir-cache-time 72h --drive-chunk-size 64M --log-level DEBUG --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off --vfs-cache-mode writes

A log from the command with the `-vv` flag (eg output from `rclone -vv copy /tmp remote:tmp`)

Here's the snippet of log around the time of freeze

2019/08/28 22:41:28 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=17383065940, fh=0x6
2019/08/28 22:41:28 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072
2019/08/28 22:41:28 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=17383197012, fh=0x6
2019/08/28 22:41:28 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072
2019/08/28 22:41:28 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=17383328084, fh=0x6
2019/08/28 22:41:57 DEBUG : /Movie: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/08/28 22:41:57 DEBUG : /Movie: >Getattr: errc=0
2019/08/28 22:41:57 DEBUG : /Movie: Opendir:
2019/08/28 22:41:57 DEBUG : /Movie: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:41:57 DEBUG : /Movie: >OpenFile: fd=Movie/ (r), err=
2019/08/28 22:41:57 DEBUG : /Movie: >Opendir: errc=0, fh=0x4
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0
2019/08/28 22:41:57 DEBUG : /Movie: Releasedir: fh=0x4
2019/08/28 22:41:57 DEBUG : /Movie: >Releasedir: errc=0
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Opendir:
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >OpenFile: fd=Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/ (r), err=
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Opendir: errc=0, fh=0x4
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Releasedir: fh=0x4
2019/08/28 22:41:57 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Releasedir: errc=0
2019/08/28 22:42:07 DEBUG : Google drive root '': Checking for changes on remote

Animosity022 · August 28, 2019, 3:41pm

Can you share the full log rather than the snippet?

ldz · August 28, 2019, 8:10pm

Here's the full log

https://mega.nz/#!k95BgKRI!y5zW4s7nDYl9xA3XTdManFWErI07jg8zihvA1iO0VDM

Animosity022 · August 28, 2019, 8:15pm

I'm not a Windows guy so not familiar with any players on Windows but whatever played you are using, is constantly opening and closing the file out.

grep Company log.log | egrep ' OpenFile: flags=O_RDONLY| Flush: fh=0x4' | wc -l
     280

You can see a bunch of these repeat over and over while you are playing it:

8/28 21:39:30 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:39:30 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Flush: fh=0x4
2019/08/28 21:39:30 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:39:30 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:39:30 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:39:30 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Flush: fh=0x4
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Flush: fh=0x4
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 21:59:20 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:29:12 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:30:08 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:30:08 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Flush: fh=0x4
2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Flush: fh=0x4
2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx

You can always try the cache backend that handles that a little better or try a different player.

ldz · August 29, 2019, 12:35am

I'm using Potplayer on Windows, I tried MPC and this issue remains.

I checked the timestamp of these "opning and closing" logs, and it matches the time when stream freezes.

I guess the player reopened the file after freezing.

The question is why the file is closed in the first place which may cause the freezing.

I do see a few ERROR logs around the time of freezing.

grep Company log.log | egrep 'ERROR'

2019/08/28 21:38:48 ERROR : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/200: read tcp 127.0.0.1:14380->127.0.0.1:1080: i/o timeout
2019/08/28 22:08:21 ERROR : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/200: read tcp 127.0.0.1:14384->127.0.0.1:1080: i/o timeout
2019/08/28 22:16:48 ERROR : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/200: unexpected EOF
2019/08/28 22:30:36 ERROR : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/200: read tcp 127.0.0.1:14920->127.0.0.1:1080: i/o timeout
2019/08/28 22:44:00 ERROR : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/200: read tcp 127.0.0.1:14969->127.0.0.1:1080: i/o timeout

I'm using a proxy to mount rclone there may be some pakcet loss, is this a probable cause?

One more question here, why there're multiple "opening and closing" logs for this file for every time the stream starts or restarts?

Is it because "--vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off --buffer-size 512M"
in my config, so rclone try to cache as much as it can?

Animosity022 · August 29, 2019, 12:47am

That's all up the application and nothing I could answer. Each application deals with file buffering and things very differently. For cloud storage though, opening and closing is expensive so you wouldn't even notice on a local drive.

Definitely not a recommended approach as that just adds another layer.

I'm not sure what you mean as the open and close is defined by the application. rclone just mounts.

Once the file is closed, the buffer drops so setting a higher buffer-size would make it worse.

The vfs items are just range requests when it gets data so really no harm in leaving them at the defaults.

ldz · August 29, 2019, 12:57am

Thanks for your reply.

After checking ERROR logs, I found somethimes rclone can recover from i/o timeout and keeps reading.
For example

2019/08/28 22:08:21 ERROR : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/200: read tcp 127.0.0.1:14384->127.0.0.1:1080: i/o timeout

2019/08/28 22:08:21 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.seek from 11734053906 to 11734053906

2019/08/28 22:08:21 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.RangeSeek from -1 to 11734053906 length -1

2019/08/28 22:08:21 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.openRange at 11734053906 length 134217728

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734053906 length 4096 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734058002 length 8192 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734066194 length 16384 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734082578 length 32768 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734115346 length 65536 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734180882 length 131072 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734311954 length 262144 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:22 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11734184978, fh=0x6

2019/08/28 22:08:22 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11734574098 length 524288 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:22 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:22 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11734316050, fh=0x6

2019/08/28 22:08:22 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:22 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11734447122, fh=0x6

2019/08/28 22:08:23 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11735098386 length 1048576 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11734578194, fh=0x6

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11734709266, fh=0x6

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11734840338, fh=0x6

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11734971410, fh=0x6

2019/08/28 22:08:23 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 11736146962 length 1048576 chunkOffset 11734053906 chunkSize 134217728

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:08:23 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=11735131438, fh=0x6

Sometimes rclone can't recover from the timeout and have to reopen the file
For example

2019/08/28 22:30:36 ERROR : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/200: read tcp 127.0.0.1:14920->127.0.0.1:1080: i/o timeout

2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Getattr: errc=0

2019/08/28 22:30:36 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.seek from 15052459669 to 15052459669

2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Open: flags=0x0

2019/08/28 22:30:36 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.RangeSeek from -1 to 15052459669 length -1

2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx

2019/08/28 22:30:36 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.openRange at 15052459669 length 134217728

2019/08/28 22:30:36 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Open: flags=O_RDONLY

2019/08/28 22:30:36 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Open: fd=Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv (r), err=

2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >OpenFile: fd=Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv (r), err=

2019/08/28 22:30:36 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Open: errc=0, fh=0x4

2019/08/28 22:30:37 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052459669 length 4096 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:37 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052463765 length 8192 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:37 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052471957 length 16384 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:37 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052488341 length 32768 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:37 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052521109 length 65536 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:37 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052586645 length 131072 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:38 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052717717 length 262144 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Getattr: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=15052590741, fh=0x6

2019/08/28 22:30:38 DEBUG : Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ChunkedReader.Read at 15052979861 length 524288 chunkOffset 15052459669 chunkSize 134217728

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Flush: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Flush: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Release: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Release: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=15052721813, fh=0x6

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: >Read: n=131072

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/The.Company.S01E02.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Read: ofst=15052852885, fh=0x6

2019/08/28 22:30:38 DEBUG : /Movie: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Opendir:

2019/08/28 22:30:38 DEBUG : /Movie: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx

2019/08/28 22:30:38 DEBUG : /Movie: >OpenFile: fd=Movie/ (r), err=

2019/08/28 22:30:38 DEBUG : /Movie: >Opendir: errc=0, fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Releasedir: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie: >Releasedir: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Opendir:

2019/08/28 22:30:38 DEBUG : /Movie: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx

2019/08/28 22:30:38 DEBUG : /Movie: >OpenFile: fd=Movie/ (r), err=

2019/08/28 22:30:38 DEBUG : /Movie: >Opendir: errc=0, fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Releasedir: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie: >Releasedir: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Opendir:

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >OpenFile: fd=Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/ (r), err=

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Opendir: errc=0, fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Releasedir: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Releasedir: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Opendir:

2019/08/28 22:30:38 DEBUG : /Movie: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx

2019/08/28 22:30:38 DEBUG : /Movie: >OpenFile: fd=Movie/ (r), err=

2019/08/28 22:30:38 DEBUG : /Movie: >Opendir: errc=0, fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: >Getattr: errc=0

2019/08/28 22:30:38 DEBUG : /Movie: Releasedir: fh=0x4

2019/08/28 22:30:38 DEBUG : /Movie: >Releasedir: errc=0

2019/08/28 22:30:38 DEBUG : /Movie/The.Company.S01.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Getattr: fh=0xFFFFFFFFFFFFFFFF

Using proxy is a must because I can't access Google Drive from my local connection.
So is there any tip for configuring rclone for this volatile network situation?

Animosity022 · August 29, 2019, 12:59am

The proxy seems to the thing slowing you down and stopping your connection. Are you able to work with who admins that and see what the issue is? There isn't a way to tune network drops that I can think of.

You could try using the rclone cache backend as that keeps some local storage and that may handle the timeouts.

Other than fixing the proxy, I don't think there's much else from the rclone side unfortunately.

ldz · August 29, 2019, 2:49am

I will monitor the connection between my local network and the proxy and do a few more tests.

But I can stream youtube 4k without any issues using this proxy.

Just out of curiosity, why is the low-level retry can succeed for some cases and fail on most cases as the above log shows?

Animosity022 · August 29, 2019, 2:58am

One is transferring HTTP through and the Youtube is built to play over the internet.

You are streaming a 4k file that is timing out multiple times and retrying and rapidly opening and closing the file.

I can upload and max my gigabit via the Google Drive Web App. I can only get about 40MB/s per file via the API. There are subtle differences in how they act.

That's more at the networking layer as it gets an answer sometimes and the retry works.

If no answer, it will retry again based on the numbers you have setup.

ldz · August 29, 2019, 3:38pm

Can you elaborate why after one retry rclone will try to reopen the file instead of requesting next chunk?

Animosity022 · August 29, 2019, 3:40pm

Apologies if I wasn't clear.

Retrying happen when something is reading a file and it has a timeout in the connection. The 'retry' parameter is what causes the number of retries.

The files being opened and closed is handled by whatever application is using the file. There is nothing you can configure in rclone in relation to this as that's how a mount works. A mount serves files and the applications call the open and close.

system · November 27, 2019, 3:54pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.