How to find corrupt data from ReadFileHandle.Read

What is the problem you are having with rclone?

I have corrupted movies/tv data in my encrypted drive which i would like to find and replace.
a lot of files have : ReadFileHandle.Read error: unexpected EOF in the tcrypt.log but they wont be there unless i interact with that file.
what i would like is a way to detect what is corrupted.

Which cloud storage system are you using? (eg Google Drive)

Google Drive

if more information is needed than i'll edit with the template.

These errors are common and normal. They occur when a stream gets paused for a while. Nothing to worry about.

The files are actually corrupted. A lot of content was being uploaded before the file was 100% moved. so as an example, a file is meant to be 3gb but only uploaded the file as 200mb.
I tried tdarr quick health check, but the file passes even though its corrupted

Ah, that's different then, but you can't really tell by that error is what I'm saying. I get these in the log all the time, and I've tracked them down to when a stream gets paused for a longer period.

ReadFileHandle.Read error: low level retry 1/10: unexpected EOF

A pause/stop can also trigger this:

ReadFileHandle.Read error: low level retry 1/10: read tcp ...:60736->...:443: wsarecv: An existing connection was forcibly closed by the remote host.

What i can do is close plex, clear the log and then try find the corrupt data, even if there is some false positives i dont mind.

If you have a lot of corrupt media, that might be hard to do. I find that a corrupt file will show a weird play time in Plex. Either way too long or way too short. I've tracked down bad files by sorting like that. Good luck!

unfortunately i dont get werid times listed in plex for the runtime.

Just to clarify, there is no command to like touch each file, or run like a health check ?

Not sure. I'd have to refer you to the rest of the gang here.

After doing a tdarr scan on one of the known corrupt files, even though the quick scan passes the file i get this in the log.
maybe i can just run a tdarr quick scan across the whole library, but worried about api bans.

2023/03/22 01:39:55 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 1: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:39:55 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:39:56 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 2: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:39:56 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:39:58 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 3: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:39:58 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:39:59 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 4: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:39:59 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:01 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 5: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:01 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:02 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 6: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:02 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:03 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 7: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:03 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:05 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 8: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:05 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:07 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 9: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:07 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:08 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 10: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:08 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:09 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 11: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:09 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:09 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: too many errors 11/10: last error: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:09 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to kick waiters: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:09 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to _ensure cache vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:09 ERROR : IO error: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:11 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 12: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:11 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:11 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: too many errors 12/10: last error: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:11 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to kick waiters: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:11 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to _ensure cache vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:11 ERROR : IO error: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:12 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 13: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:12 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:12 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: too many errors 13/10: last error: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:12 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to kick waiters: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:12 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to _ensure cache vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:12 ERROR : IO error: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:14 INFO  : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: downloader: error count now 14: vfs reader: failed to write to cache file: unexpected EOF
2023/03/22 01:40:14 ERROR : tv/South Park (1997)/Season 25/South Park (1997) - S25E02 - The Big Fix [WEBDL-1080p][AC3 5.1][x264]-NTb.mkv: vfs cache: failed to download: vfs reader: failed to write to cache file: unexpected EOF

Rclone itself doesn't do partial uploads so you'd want to figure out how you created a partial upload if something didn't get completed.

I think he meant that Rclone completed a full upload, but of a file that had not been moved fully to its (local) upload location.

This is why I do everything manually :wink:

It wouldn't be partial then at that point. I use cache full and regardless of it is local or uploaded to the remote, you should have the same behavior.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.