Retry errors and "I/O operation has been aborted because of either a thread exit or an application request"

TCCalvin1 · June 10, 2020, 12:55pm

What is the problem you are having with rclone?

Multiple retry errors and "I/O operation has been aborted" along with "The system cannot find the file specified" (even when the file is there and no issues with LAN)

What is your rclone version (output from `rclone version`)

rclone v1.52.0

os/arch: windows/amd64
go version: go1.14.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 x64

Which cloud storage system are you using? (eg Google Drive)

Google Drive (Gsuite)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone sync \\local.ip.address\shared.drive E:\mounted.google.drive -P --no-update-modtime --size-only --exclude-from c:/rclone/rcloneexclude.txt

The rclone config contents with secrets removed.

[Gdrive]
type = drive
scope = drive
token = {"access_token":"xxxxxxxx","token_type":"Bearer","refresh_token":"xxxxx","expiry":"2020-06-10T13:20:08.8009859+01:00"}
root_folder_id = xxxxxxxxxxxxx

Animosity022 · June 10, 2020, 12:57pm

Can you please share a debug log as that is need to help you out.

TCCalvin1 · June 12, 2020, 10:18am

Apologies, the log is huge and contains customer names/details so I've snipped a bit out. The errors are just multiple repeats of the below.

If I run the rclone command again, eventually everything syncs (sometimes takes 3-4 repeats).

During the process when errors are present, rclone fails to delete the fails on the target also. If it completes without the errors, it deletes the files without issue.

2020/06/12 10:01:48 DEBUG : rclone: Version "v1.52.0" starting with parameters ["rclone" "sync" "\\192.168.0.50\Local.dir" "E:\mounted.google.drive" "-P" "--no-update-modtime" "--size-only" "--exclude-from" "c:/rclone/rcloneexclude.txt" "-vv" "--log-file=C:\Users\User\Desktop\rclonelog.txt"]
2020/06/12 10:01:48 DEBUG : Using config file from "C:\Users\User\.config\rclone\rclone.conf"
2020/06/12 10:01:48 DEBUG : fs cache: renaming cache item "\\192.168.0.50\Local.dir" to be canonical "//?/UNC/192.168.0.50/Local.dir"
2020/06/12 10:01:49 DEBUG : fs cache: renaming cache item "E:\mounted.google.drive" to be canonical "//?/E:/mounted.google.drive"
2020/06/12 10:01:49 DEBUG : $RECYCLE.BIN: Excluded
2020/06/12 10:01:49 DEBUG : System Volume Information: Excluded
2020/06/12 10:01:49 DEBUG : _SYNCAPP: Excluded
2020/06/12 10:01:50 DEBUG : System Volume Information: Excluded
2020/06/12 10:01:50 DEBUG : _SYNCAPP: Excluded
2020/06/12 10:01:57 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:01:57 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)

2020/06/12 10:34:43 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)
2020/06/12 10:38:10 ERROR : DirectoryName/xxxxxxxxxxxx.ext: Failed to copy: chtimes \?\E:\DirectoryName/xxxxxxxxxxxx.ext: The system cannot find the file specified.
2020/06/12 10:38:13 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:38:13 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)
2020/06/12 10:38:18 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:38:18 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)
2020/06/12 10:39:21 ERROR : DirectoryName/xxxxxxxxxxxx.ext: Failed to copy: chtimes \?\E:\DirectoryName/xxxxxxxxxxxx.ext: The system cannot find the file specified.
2020/06/12 10:39:25 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:39:25 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)
2020/06/12 10:39:34 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:39:34 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)
2020/06/12 10:41:55 ERROR : DirectoryName/xxxxxxxxxxxx.ext: Failed to copy: chtimes \?\E:\DirectoryName/xxxxxxxxxxxx.ext: The system cannot find the file specified.
2020/06/12 10:42:00 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:42:00 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)
2020/06/12 10:42:05 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:42:05 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)
2020/06/12 10:42:42 ERROR : DirectoryName/xxxxxxxxxxxx.ext: Failed to copy: chtimes \?\E:\DirectoryName/xxxxxxxxxxxx.ext: The system cannot find the file specified.
2020/06/12 10:42:46 DEBUG : DirectoryName/xxxxxxxxxxxx.ext: MD5 = xxxxxxxxxxxxxx OK
2020/06/12 10:42:46 INFO : DirectoryName/xxxxxxxxxxxx.ext: Copied (new)

Animosity022 · June 12, 2020, 10:53am

I'd think the issue relates to the mount and being unable to update the mod times properly based on how you have the mount running.

In general, it's better/faster/less prone to issues to copy to the mount rather than mounting to copy.

Change the second part of the copy to go to Gdrive: instead of using E: would be good anyway.

If you want to figure out the mount as well, I think we need to know how you mounted the drive.

TCCalvin1 · June 12, 2020, 11:37am

Thanks for the quick response.

Drive is mounted as below:

rclone mount Gdrive: E: --vfs-cache-mode writes --cache-dir I: --checkers 3 --tpslimit 3 --transfers 3 --drive-chunk-size 64M --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit off

Animosity022 · June 12, 2020, 11:46am

Not the actual mount command, it's the way it's mounted on windows as admin/mssm, not my thing, perhaps @asdffdsa can chime in.

asdffdsa · June 12, 2020, 1:04pm

hello,

what version of winfsp are you running?
how are you running the rclone mount command?
command line, nssm, task scheduler
which username are you running the mount command as?
why not copy direct to gdrive and not use the e: mount?

ncw · June 12, 2020, 3:02pm

This is undoubtedly a mount bug as @Animosity022 was too polite to say Copying direct to drive will be more reliable rather than through the mount - it go much faster and be more reliable. This bug I'm pretty sure is fixed with the new VFS code though.

So my advice is use rclone copy direct to drive...

TCCalvin1 · June 12, 2020, 4:00pm

Thanks for all of the replies, the support here is unbelievably good (and newb friendly). I've only been using rclone for a few days and can already see how good it is.

I've switched to copying direct and so far so good (touch wood), it has been chugging away for almost an hour now with 0 errors.

ncw · June 12, 2020, 4:15pm

that will work fine (I'm sure it will!). move/copy/sync is rclone's bread and butter.

system · August 12, 2020, 12:15pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.