Sporadic non-sequential `ReadFileHandle.Read error: low level retry 1/10: EOF` error

Help and Support
1

What is the problem you are having with rclone?

Random ReadFileHandle.Read error: low level retry 1/10: EOF error when a process such as Plex is reading the drive. This started happening only today and never happened before.

From my research this error is probably the fault on Google Server that causes the stream to suddenly terminate. The enlightening news is those retry stop at 1/10 and not sequential, so I'm guessing it went away in the second retry.

I'm just trying to get assurance here that those files I previously uploaded is still okay and I don't have to do a hash checking again. Or perhaps I'm being too paranoid.

As a resort I completely removed the cache backend and just using [gdrive] -> [gcrypt] pipeline for now to avoid any problem caused by the cache. And voila those EOF error never occured again, at least right now. I'm going to use up the 750G daily quota and see if the same problem resurface.

I did some reading and realized that with the cache backend active, e.g. [gdrive] -> [gcache] -> [gcrypt], the vfs settings won't take into effect. Also the cache backend might be obsolete in the future rClone release. Need confirmation here.

What is your rclone version (output from rclone version)

rclone v1.52.2
- os/arch: linux/amd64
- go version: go1.14.4

Which OS you are using and how many bits (eg Windows 7, 64 bit)

4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Which cloud storage system are you using? (eg Google Drive)

GDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone-out.txt contains the list of dir filter I manually specified.

rclone mount gcrypt: ~/mnt/gdrive --allow-other --allow-non-empty --cache-db-purge --buffer-size 128M --use-mmap --dir-cache-time 72h --drive-chunk-size 256M --timeout 1h --vfs-cache-mode writes --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit 1G --no-modtime -v

rclone copy /mnt/downloads/media-staging ~/mnt/gdrive/test --include-from rclone-out.txt -P

The rclone config contents with secrets removed.

[gdrive]
type = drive
scope = drive
token = {"access_token":"**redacted**"}
root_folder_id = **redacted**
client_id = **redacted**

[gcache]
type = cache
remote = gdrive:/gdrive
chunk_size = 5M
info_age = 1d
chunk_total_size = 10G

[gcrypt]
type = crypt
remote = gcache:/crypt
filename_encryption = standard
directory_name_encryption = true
password = **redacted**
password2 = **redacted**

A log from the command with the -vv flag

2020/08/07 16:09:56 ERROR : test/Hatchet II (2010)/Hatchet.II.2010.BluRay.1080p.DTS-HD.MA.5.1.AVC.REMUX-Jerrk.mkv: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2020/08/07 16:13:19 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/lt4q8dcaa0hd3hblatcbmmmql8/g1l5vcnrkj7deht45v732bgciq88ne3k0gvbinbdemetr5c1gu75ojhqv3rhbl6shvah1dni5ggaldlis0k2k5fni6c72r2c8k8cdtavisolk4ltt9o4uc89kkgjfrtp: (8629780480/23682811227) error (chunk not found 8629780480) response
2020/08/07 16:13:19 ERROR : test/Harpoon (2019)/Harpoon 2019 1080p Blu-ray Remux AVC DTS-HD MA 5.1 - KRaLiMaRKo.mkv: ReadFileHandle.Read error: low level retry 1/10: EOF
2020/08/07 16:16:34 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/bfsonvmec37oke132ja44pnl4ttg8gu1rbpc87inchcj35s2rgbg/08ln5dbjdrumoqs3dblbaft8p2vfhsb6b7giqlnqse66j6g1qn0m82h91tao9o9sa7f2mqrbsj6s12o7t322r5dcjr1ct4ref4q2or6eb7va9hrq45i45buni62g8j68: (9615441920/22850281916) error (chunk not found 9615441920) response
2020/08/07 16:16:34 ERROR : test/Happy Death Day 2U (2019)/Happy Death Day 2U 2019 1080p Blu-ray Remux AVC DTS-HD MA 5.1 - KRaLiMaRKo.mkv: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2020/08/07 16:17:29 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/ss2lcs9528fhtdm61i2k0umufs/rj4f0q5pfs176t82g8u8ngmic7gvc0baupbod1hg2ma5o0s4tivq6g5sdi02vvlu870kd1517hh8o52rc0l8lenef20i3eebju424hg: (9678356480/17584089199) error (chunk not found 9678356480) response
2020/08/07 16:17:29 ERROR : test/Hatchet (2006)/Hatchet.2006.BluRay.1080p.DTS-HD.MA.7.1.AVC.REMUX-FraMeSToR.mkv: ReadFileHandle.Read error: low level retry 1/10: EOF
2020/08/07 16:27:50 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/ss2lcs9528fhtdm61i2k0umufs/rj4f0q5pfs176t82g8u8ngmic7gvc0baupbod1hg2ma5o0s4tivq6g5sdi02vvlu870kd1517hh8o52rc0l8lenef20i3eebju424hg: (12567183360/17584089199) error (chunk not found 12567183360) response
2020/08/07 16:27:50 ERROR : test/Hatchet (2006)/Hatchet.2006.BluRay.1080p.DTS-HD.MA.7.1.AVC.REMUX-FraMeSToR.mkv: ReadFileHandle.Read error: low level retry 1/10: EOF
2020/08/07 16:39:17 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/iamue5n1hj4nccgl6ef6b6k571n9d4b20qjqt76oj8e66il19te0/42e8f4telqfh4hif7hbv2va0nvqtnn5siopbamlhkl4bh9o9aien2k9ck2k1fbvav6t21ivba8n1e5envqg96c5hrf4md6merbk373g: (15885926400/17499789106) error (chunk not found 15885926400) response
2020/08/07 16:39:17 ERROR : test/Hatchet II (2010)/Hatchet.II.2010.BluRay.1080p.DTS-HD.MA.5.1.AVC.REMUX-Jerrk.mkv: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2020/08/07 16:47:24 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/mj66nd7ju6jn0sfhc7073284so/oi1omer3ur0mvpipg7rmnqbfp5d5kbglt0lpi3pjnpj4k5g1lclh465e0dkrt0dve9opfdnp0njcfo8ev2mqu5sqej0pnhkdrpgld68l72oho2nro3nv9s26jtssrugt: (17820549120/25126813910) error (chunk not found 17820549120) response
2020/08/07 16:47:24 ERROR : test/Haunt (2019)/Haunt 2019 1080p Blu-ray Remux AVC DTS-HD MA 5.1 - KRaLiMaRKo.mkv: ReadFileHandle.Read error: low level retry 1/10: EOF
2020/08/07 16:55:55 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/mj66nd7ju6jn0sfhc7073284so/oi1omer3ur0mvpipg7rmnqbfp5d5kbglt0lpi3pjnpj4k5g1lclh465e0dkrt0dve9opfdnp0njcfo8ev2mqu5sqej0pnhkdrpgld68l72oho2nro3nv9s26jtssrugt: (20462960640/25126813910) error (chunk not found 20462960640) response
2020/08/07 16:55:55 ERROR : test/Haunt (2019)/Haunt 2019 1080p Blu-ray Remux AVC DTS-HD MA 5.1 - KRaLiMaRKo.mkv: ReadFileHandle.Read error: low level retry 1/10: EOF
2020/08/07 16:57:57 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/6j0basic0fvkp0t751fpcaeoge630uq0jfi1imc8irqs8ij679bisdhmtom0p0a86ccnfbglk2vai/8uimb7kq7sl3916fn9iu3eru1onnmb87k5celksph8qt60tu0jparq44vve9non9883c5lbdm1v3cmgpebf68agor1lkut5m15e3fackdldsdm5rcrj37l8fhu65qhfhtn9fppejlemvrl4n9ujdv95c3qbckmptctj00r8iekcn9745ts90: (20898119680/68122093538) error (chunk not found 20898119680) response
2020/08/07 16:57:57 ERROR : test/Harry Potter and the Prisoner of Azkaban (2004)/Harry Potter and the Prisoner of Azkaban 2004 2160p UHD Blu-ray Remux HEVC DTS-X 7.1 - BluDragon.mkv: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2020/08/07 17:01:27 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/bfsonvmec37oke132ja44pnl4ttg8gu1rbpc87inchcj35s2rgbg/08ln5dbjdrumoqs3dblbaft8p2vfhsb6b7giqlnqse66j6g1qn0m82h91tao9o9sa7f2mqrbsj6s12o7t322r5dcjr1ct4ref4q2or6eb7va9hrq45i45buni62g8j68: (22476226560/22850281916) error (chunk not found 22476226560) response
2020/08/07 17:01:27 ERROR : test/Happy Death Day 2U (2019)/Happy Death Day 2U 2019 1080p Blu-ray Remux AVC DTS-HD MA 5.1 - KRaLiMaRKo.mkv: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2020/08/07 17:05:55 ERROR : crypt/d28g95jaknfnupokvl1u3tvd2c/kb62pv9558jm4qn22arpje8vi4pm24ksu27t6n63saqu7kea5ti0/iagrtfmbdpufkb8011r02ftinon3ee19523pul1sc98leiihlo3jk0jdrph64uf0eossk604rpco66smqvoo8qn3490tjlhju5686n9bdgf3t151rej44k1grus5ilag: (23870832640/25559521189) error (chunk not found 23870832640) response
2020/08/07 17:05:55 ERROR : test/Happy Death Day (2017)/Happy.Death.Day.2017.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: ReadFileHandle.Read error: low level retry 1/10: EOF
1 Like
2

hello and welcome to the forum,

i see that you are using the cache backend, and i suggest that you remove it from your config.
https://rclone.org/cache/#status

also, the experts will tell you that --allow-non-empty is not a good idea.

try using a little flags as possible, the more flags, the more problems.

3

Hey there, yup I've since removed the cache backend after the EOF error. Since I always make sure the mount point is empty prior to rclone mount, the allow-non-empty doesn't really matter. But I will remove it in my future mount command just in case.

[gdrive]
type = drive
scope = drive
token = {"access_token":"**redacted**"}
root_folder_id = **redacted**
client_id =**redacted**
client_secret =**redacted**

[gcrypt]
type = crypt
remote = gdrive:/gdrive/crypt
filename_encryption = standard
directory_name_encryption = true
password = **redacted**
password2 = **redacted**
4
  • are removing the cache, what is your mount command. the less flags the better?
  • not sure it is a good idea, to rclone copy to a rclone mount?
    why not just rclone copy direct to gdrive remote?
5

It's a copy from my local drive which is mounted at system /mnt/downloads/media-staging to my rclone remote mounted at my user home dir ~/mnt/gdrive. Notice the / and `~' if that is not clear :slight_smile:

Those flags are what I found to be optimal for my current 1Gbps uplink, or so I believe since my uplink is nearly 100% utilized during those rclone copy.

6

i could be wrong, but never saw anybody rclone copy to a rclone mount.
in effect, using rclone twice instead of once, double rclone trouble?
no need for the mount, just copy direct to gcrypt:?

perhaps
rclone copy /mnt/downloads/media-staging gcrypt:test
instead of
rclone copy /mnt/downloads/media-staging ~/mnt/gdrive/test

7

Ah yes I see your point now. Strangely enough I have been doing like that for almost a month now and all the files are encrypted and decrypted properly. Not sure will doing it my way will pose a potential problem?

9

seems that you have real problems, using your setup given your need to post here.
not sure how to reply except to suggest to simply your setup in terms of commands and flags.

10

The "unexpected EOF" error is actually very common. It can happen when a video stream gets paused. Has nothing to do with cache or even with Plex. I don't use Plex personally, but I still see this error every once in a while on my HTPC. On my Plex server, with multiple users streaming every day all day, using rclone mount -v, I see this error much more frequently. Nothing to worry about, in my opinion.

1 Like
11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.