Some errors wich might be bugs?

Suspected Bug
1

I am currently facing some errors in the log file which might not change the stability of rclone but still are making me nervous.

Error 1: WriteFileHandle: ReadAt: Can't read and write to file without --vfs-cache-mode >= minimal
Error 2: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes

As you can see I configured the setting "full" for the cache, so I do not get the point here.

Error 3: vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file from cache to remote: multipart upload failed to read source: can't copy - source file is being updated (mod time changed from 2020-11-12 16:38:54.951599998 +0100 CET to 2020-11-12 16:39:02.277442834 +0100 CET)
Error 4: Failed to copy: can't copy - source file is being updated (mod time changed from 2020-11-12 16:39:54.360048186 +0100 CET to 2020-11-12 16:39:59.506139977 +0100 CET)
Error 5: vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file from cache to remote: can't copy - source file is being updated (mod time changed from 2020-11-12 16:39:54.360048186 +0100 CET to 2020-11-12 16:39:59.506139977 +0100 CET)

So I know its not irrelevant that the mod time changed in cache but I am not quite sure why this is an error. I also do not get the point here.

Rclone Settings:

--daemon
--cache-dir /mnt/resource/rclonecache
--track-renames
--dir-cache-time 24h
--attr-timeout 4m
--vfs-write-back 5s
--vfs-cache-mode full
--transfers=32
--azureblob-chunk-size 100m
--checksum
--update
--multi-thread-cutoff=32m
--multi-thread-streams=32
--vfs-cache-max-age 168h
--vfs-cache-max-size 200g
--allow-other
--allow-non-empty

What is your rclone version (output from rclone version)

1.53.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux 64bit

Which cloud storage system are you using? (eg Google Drive)

Azure Blob

Thanks

2

You shouldn't be getting these errors with --vfs-cache-mode full.

Can you double check your config. If you run rclone with -vv then it will print what flags it is actually using at the start of the log.

If it is still a problem can you give instructions on how to reproduce please.

This is rclone saying it was uploading a file, but it changed so it abandoned the upload as it didn't want to upload a corrupted file.

What should have happened is that rclone cancelled the upload when it was written I think.

Although the time difference between these is very small

2020-11-12 16:39:54.360048186 +0100 CET
2020-11-12 16:39:59.506139977 +0100

What filesystem is your local machine? Some filesystems don't keep the time constant. There is a flag to disable the check

  --local-no-check-updated           Don't check to see if the files change during upload
3

I checked the the debug log and it says:
2020/11/16 16:29:03 DEBUG : rclone: Version "v1.53.2" starting with parameters ["/usr/bin/rclone" "mount" XXXXX "--daemon" "--cache-dir" "/mnt/resource/rclonecache" "--track-renames" "--dir-cache-time" "24h" "--attr-timeout" "4m" "--vfs-write-back" "5s" "--vfs-cache-mode" "full" "--transfers=32" "--azureblob-chunk-size" "100m" "--checksum" "--update" "--multi-thread-cutoff=32m" "--multi-thread-streams=32" "--vfs-cache-max-age" "168h" "--vfs-cache-max-size" "200g" "--log-level=DEBUG" "--allow-other" "--allow-non-empty" "--use-mmap"

Unfortunatly, or fortunatly :wink: it stopped 2 days ago, so I cannot say how to reproduce it. As soon as the error happens again I will debug it.

Isnt this expected behavior? A file is written to rclone cache disk, after the wait time it start to upload the files and if the file is updated during the upload it should cancel it and try to upload next time the wait time is reached?

I am using ext4.

4

Great :slight_smile:

Yes that is what does happen.

OK, that is unlikely to be the time change problem then!

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.