Recent change to reporting EOF errors

Dev Discussions
1

@ncw

commit d5afcf9e34044e5b7bb1ae5553b6f2f98450b7d2
Author: Nick Craig-Wood nick@craig-wood.com
Date: Mon Mar 6 17:49:50 2023 +0000

crypt: try not to return "unexpected EOF" error

Before this change the code wasn't taking into account the error
io.ErrUnexpectedEOF that io.ReadFull can return properly. Sometimes
that error was being returned instead of a more specific and useful
error.

To fix this, io.ReadFull was replaced with the simpler
readers.ReadFill which is much easier to use correctly.

Nick, this recent change turned the normal one-liner EOFs I receive whenever a streamed file is paused for too long

2023/03/25 22:28:18 ERROR : [...]: ReadFileHandle.Read error: low level retry 1/10: read tcp [...] i/o timeout

into this

2023/03/25 22:28:18 ERROR : [...]: ReadFileHandle.Read error: low level retry 1/10: read tcp [...] i/o timeout
2023/03/25 22:28:18 ERROR : [...]: ReadFileHandle.Read error: low level retry 2/10: EOF
2023/03/25 22:28:18 ERROR : [...]: ReadFileHandle.Read error: low level retry 3/10: EOF
2023/03/25 22:28:19 ERROR : [...]: ReadFileHandle.Read error: low level retry 4/10: EOF
2023/03/25 22:28:19 ERROR : [...]: ReadFileHandle.Read error: low level retry 5/10: EOF
2023/03/25 22:28:19 ERROR : [...]: ReadFileHandle.Read error: low level retry 6/10: EOF
2023/03/25 22:28:19 ERROR : [...]: ReadFileHandle.Read error: low level retry 7/10: EOF
2023/03/25 22:28:19 ERROR : [...]: ReadFileHandle.Read error: low level retry 8/10: EOF
2023/03/25 22:28:20 ERROR : [...]: ReadFileHandle.Read error: low level retry 9/10: EOF
2023/03/25 22:28:20 ERROR : [...]: ReadFileHandle.Read error: low level retry 10/10: EOF
2023/03/25 22:28:20 ERROR : [...]: ReadFileHandle.Read error: EOF
2023/03/25 22:28:20 ERROR : [...]: ReadFileHandle.Release error: file already closed
2023/03/25 22:28:20 ERROR : IO error: file already closed

Was that intended?

2

Hmm now it is retrying the errors which is sensible, but the EOF error is not sensible .

It is supposed to be re-opening the file for a low level retry which should sort out the problem but it looks like that isn't working for some reason.

Did it used to sort the stream out before after a long pause and now it doesn't?

If you revert d5afcf9e34044e5b7bb1ae5553b6f2f98450b7d2 on master does it fix the problem?

I'm not quite sure what is going on here. Can you describe a way for me to reproduce this?

Thanks

3

I don't think anything has changed in the way Rclone works in that regard. It seems as if it's simply outputting more info now. These errors have always shown in the console, but never had any impact on the streams. Treat this as an observation rather than a bug report :slight_smile:

I'll keep you posted if I notice anything else.

4

Ok thanks. I'll have another look at the code tomorrow just to double check.

1 Like
5

Well I had another look and I can't figure it out! If it becomes a problem then please let me know!

6

Will do. I really think it's just spitting out additional info that it didn't before. The retries (2-10), basically.

1 Like
7

commit bb6cfe109d7295c0c27a664dd6226cd38f7d40bb
Author: Nick Craig-Wood nick@craig-wood.com
Date: Mon Apr 24 16:24:22 2023 +0100

crypt: fix reading 0 length files

In an earlier patch

d5afcf9e34044e5b crypt: try not to return "unexpected EOF" error

This introduced a bug for 0 length files which this fixes which only
manifests if the io.Reader returns data and EOF which not all readers
do.

This was failing in the integration tests.

This commit seems to have reverted back to the former way of only reporting the first retry in the console :slight_smile:

8

Good, I think it is working properly now! I reviewed the code about 3 times :crossed_fingers:

Thanks for testing.

1 Like
9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.