IO Error - Non-encrypted GDrive

subpar · April 26, 2020, 4:07pm

What is the problem you are having with rclone?

IO errors in log, concern?

What is your rclone version (output from `rclone version`)

1.51.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 8.1

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone.exe mount GDRIVE:/ Y: --vfs-cache-mode writes

A log from the command with the `-vv` flag (eg output from `rclone -vv copy /tmp remote:tmp`)

X:\rclone-v1.51.0-windows-amd64>rclone.exe mount GDRIVE:/ Y: --vfs-cache-mode writes
The service rclone has been started.
2020/04/26 10:57:57 ERROR : C1806D10: Failed to calculate src hash: hash: failed to stat: CreateFile \\?\C:\Users\*****\
AppData\Local\rclone\vfs\GDRIVE\C1806D10: The system cannot find the file specified.
2020/04/26 10:57:57 ERROR : C1806D10: corrupted on transfer: MD5 hash differ "" vs "3cf0d61ee613a5549dadb95318c85ee6"
2020/04/26 10:57:58 ERROR : Book1.xlsx(0xc00011a180): failed to transfer file from cache to remote: corrupted on transfe
r: MD5 hash differ "" vs "3cf0d61ee613a5549dadb95318c85ee6"
2020/04/26 10:57:58 ERROR : Book1.xlsx(0xc00011a180): RWFileHandle.Flush error: failed to transfer file from cache to re
mote: corrupted on transfer: MD5 hash differ "" vs "3cf0d61ee613a5549dadb95318c85ee6"
2020/04/26 10:57:58 ERROR : IO error: failed to transfer file from cache to remote: corrupted on transfer: MD5 hash diff
er "" vs "3cf0d61ee613a5549dadb95318c85ee6"
2020/04/26 10:58:16 ERROR : ~$Book1.xlsx(0xc00011a080): failed to find cache file: permission denied
2020/04/26 10:58:16 ERROR : ~$Book1.xlsx(0xc00011a080): RWFileHandle.Flush error: failed to find cache file: permission
denied
2020/04/26 10:58:16 ERROR : IO error: failed to find cache file: permission denied
2020/04/26 10:58:16 ERROR : ~$Book1.xlsx(0xc00011a080): failed to find cache file: object not found
2020/04/26 10:58:16 ERROR : ~$Book1.xlsx(0xc00011a080): RWFileHandle.Release error: failed to find cache file: object no
t found
2020/04/26 10:58:16 ERROR : IO error: failed to find cache file: object not found

Animosity022 · April 26, 2020, 5:09pm

Looks like you don't have permissions to something so that would be a problem since it's trying to write to it.

Check out:

https://rclone.org/commands/rclone_mount/#windows-caveats for some tips on how to mount it properly in Windows.

asdffdsa · April 26, 2020, 5:13pm

what version of winfsp are you using?
the lastest stable is v1.6

you might try running the rclone mount as system user, using task scheduler.

subpar · April 26, 2020, 7:18pm

Using latest winsfp (1.6) - just installed yesterday.

I am running as normal user - not running .bat as Administrator. Once I could see things working OK was then planning to create a service for it (nssm).

asdffdsa · April 26, 2020, 7:46pm

you can create a task in task scheduler in about one minute as system user.
it is good for quick prototyping and starting tasks on demand.

ncw · April 27, 2020, 9:25am

I think you are trying to edit google docs in an rclone mount - is that correct?

It looks like rclone uploaded your xlsx as a google doc which is a little surprising but perhaps it was an xlsx doc originally?

subpar · April 27, 2020, 5:04pm

I am trying to eventually use a rclone mounted (and encrypted) drive in windows as a NAS. Create and store files - similar to what I use Spideroak Hive (though this rclone encrypted NAS has no "sync")

Yes - file was a test.xlsx created locally and saved to the mounted rclone on Gdrive (unencrypted). That said, I see the same "noise" in the command window as well when I was trying encrypted.

I should note, the creation, editing and saving of the file seemed to work despite the verbose protest in the command window.

Thanks again for the comments - would love to see if there is some guidance on fixing or is rclone just a poor tool for my intended use case (using an encrypted rclone mount on Gdrive as a NAS impossible/impractical)

ncw · April 28, 2020, 4:27pm

OK

Can you post that log too?

That is good news! I think there might be a bug lurking there but I'm not sure what is going on.

I'd like to see if there is a bug that needs fixing!

As to whether it will work for your use case - maybe! The filing system presented by rclone is reasonably compatible with applications now-a-days but not 100% as a remote object storage system is quite different to the disk storage that filing systems expect.

I'm actively working on this at the moment!

subpar · April 29, 2020, 3:43pm

Created and saved new file (rclone.xlsx) and closed excel. Browsed to the mounted file in Explorer, re-opened the saved file to be sure it was OK (it was).

Here is the log (couldn't upload attachment as a new user)

2020/04/29 10:37:47 INFO  : Cleaned the cache: objects 2 (was 0), total size 8.457k (was 0)
The service rclone has been started.
2020/04/29 10:38:33 INFO  : rclone.xlsx: Copied (new)
2020/04/29 10:38:34 INFO  : rclone.xlsx: Removed from cache
2020/04/29 10:38:35 INFO  : 3DA1CC20: Renamed in cache
2020/04/29 10:38:36 ERROR : 3DA1CC20: Failed to calculate src hash: hash: failed to stat: CreateFile \\?\C:\Users\JWU42\AppData\Local\rclone\vfs\GDRIVE\3DA1CC20: The system cannot find the file specified.
2020/04/29 10:38:36 ERROR : 3DA1CC20: corrupted on transfer: MD5 hash differ "" vs "c0ba5ed97d055c5f7be950f8cfb01d9a"
2020/04/29 10:38:36 INFO  : 3DA1CC20: Removing failed copy
2020/04/29 10:38:37 ERROR : rclone.xlsx(0xc000262140): failed to transfer file from cache to remote: corrupted on transfer: MD5 hash differ "" vs "c0ba5ed97d055c5f7be950f8cfb01d9a"
2020/04/29 10:38:37 ERROR : rclone.xlsx(0xc000262140): RWFileHandle.Flush error: failed to transfer file from cache to remote: corrupted on transfer: MD5 hash differ "" vs "c0ba5ed97d055c5f7be950f8cfb01d9a"
2020/04/29 10:38:37 ERROR : IO error: failed to transfer file from cache to remote: corrupted on transfer: MD5 hash differ "" vs "c0ba5ed97d055c5f7be950f8cfb01d9a"
2020/04/29 10:38:38 INFO  : rclone.xlsx: Copied (new)
2020/04/29 10:38:41 INFO  : ~$rclone.xlsx: Removed from cache
2020/04/29 10:38:41 ERROR : ~$rclone.xlsx(0xc0000785c0): failed to find cache file: permission denied
2020/04/29 10:38:41 ERROR : ~$rclone.xlsx(0xc0000785c0): RWFileHandle.Flush error: failed to find cache file: permission denied
2020/04/29 10:38:41 ERROR : IO error: failed to find cache file: permission denied
2020/04/29 10:38:41 ERROR : ~$rclone.xlsx(0xc0000785c0): failed to find cache file: object not found
2020/04/29 10:38:41 ERROR : ~$rclone.xlsx(0xc0000785c0): RWFileHandle.Release error: failed to find cache file: object not found
2020/04/29 10:38:41 ERROR : IO error: failed to find cache file: object not found
2020/04/29 10:38:47 INFO  : Cleaned the cache: objects 4 (was 4), total size 16.910k (was 8.457k)
2020/04/29 10:38:53 INFO  : ~$rclone.xlsx: Removed from cache
2020/04/29 10:38:53 ERROR : ~$rclone.xlsx(0xc00036e640): failed to find cache file: permission denied
2020/04/29 10:38:53 ERROR : ~$rclone.xlsx(0xc00036e640): RWFileHandle.Flush error: failed to find cache file: permission denied
2020/04/29 10:38:53 ERROR : IO error: failed to find cache file: permission denied
2020/04/29 10:38:53 ERROR : ~$rclone.xlsx(0xc00036e640): failed to find cache file: object not found
2020/04/29 10:38:53 ERROR : ~$rclone.xlsx(0xc00036e640): RWFileHandle.Release error: failed to find cache file: object not found
2020/04/29 10:38:53 ERROR : IO error: failed to find cache file: object not found
The service rclone has been stopped.

ncw · April 30, 2020, 7:48am

Thanks for that.

I see what is going on now - it looks like a race between the renaming of the file and the upload of the file.

The new delayed writeback will fix this in due course...

subpar · April 30, 2020, 6:21pm

Thank you for the coming support!

system · June 30, 2020, 2:21pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.