Slew of input/output errors when syncing

Help and Support
1

What is the problem you are having with rclone?

I've done a half dozen sync runs with rclone, and each time I'm having a number of the files fail to sync with input/output errors. It's different files each time that fail, so I can't seem to find a pattern. It doesn't appear to be due to the path length or file type. The sync runs for several hours due to the number of files in the source directory, and most files sync (or determine they don't need to) without a problem, but I'm getting 30-40 file failures each time.

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1

  • os/version: darwin 12.4 (64 bit)
  • os/kernel: 21.5.0 (x86_64)
  • os/type: darwin
  • os/arch: amd64
  • go/version: go1.18.1
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

A Windows fileserver running Acronis Connect so that I can mount the drive as AFP instead of SMB.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -vv --log-file=/Users/jpollock/Scripts/rsync/logs/rclonelog sync ~/Documents/Job-related/PJC/RTC\ stuff/NCCRC/ ~/Scripts/rsync/vdrive/NCCRC/ --exclude MariaFolder/ --exclude AmandaFolder/ --links

The rclone config contents with secrets removed.

Not using a config file; I'm relying on the defaults.

A log from the command with the -vv flag

The log is way too long to paste the entire contents, but here are some of the relevant entries:

2022/06/07 21:29:51 ERROR : Media/Citations to NCCRC/As Evictions Loom, Lawyers Are Gearing Up to Help (New York Times).pdf: Failed to calculate dst hash: hash: failed to read: read /Users/jpollock/Scripts/rsync/vdrive/NCCRC/Media/Citations to NCCRC/As Evictions Loom, Lawyers Are Gearing Up to Help (New York Times).pdf: input/output error
2022/06/07 21:30:48 ERROR : Conferences or presentations/Master presentations and outlines/CRTC as movement for turbulent times.pptx: Failed to calculate dst hash: hash: failed to read: read /Users/jpollock/Scripts/rsync/vdrive/NCCRC/Conferences or presentations/Master presentations and outlines/CRTC as movement for turbulent times.pptx: input/output error
2022/06/07 21:33:10 ERROR : Active efforts/Active research/Things to review/Children's Right to Counsel - Statutory Reform, Litigation, Delivery Systems.mp4: Failed to calculate dst hash: hash: failed to read: read /Users/jpollock/Scripts/rsync/vdrive/NCCRC/Active efforts/Active research/Things to review/Children's Right to Counsel - Statutory Reform, Litigation, Delivery Systems.mp4: input/output error
2022/06/07 21:34:59 ERROR : Active efforts/Pending legislation or rules changes/dead bills/2021/TX HB 2680 (mandatory services for parent).pdf: Failed to calculate dst hash: hash: failed to read: read /Users/jpollock/Scripts/rsync/vdrive/NCCRC/Active efforts/Pending legislation or rules changes/dead bills/2021/TX HB 2680 (mandatory services for parent).pdf: input/output error
2022/06/07 22:30:01 ERROR : Local file system at /Users/jpollock/Scripts/rsync/vdrive/NCCRC/: not deleting files as there were IO errors
2022/06/07 22:30:01 ERROR : Local file system at /Users/jpollock/Scripts/rsync/vdrive/NCCRC/: not deleting directories as there were IO errors
2022/06/07 22:30:01 ERROR : Attempt 1/3 failed with 11 errors and: hash: failed to read: read /Users/jpollock/Scripts/rsync/vdrive/NCCRC/Active efforts/Expansion efforts/Housing work/COVID-19/moratoria/federal/CARES Act/Violations of moratorium/lawsuits/Virginia federal lawsuit for violation of CARES Act.pdf: input/output error
2022/06/07 22:30:12 ERROR : Legal research and initiatives/non-subject-specific research/studies and reports/Houseman 2019 Civil Legal Aid Report.pdf: Failed to calculate dst hash: hash: failed to read: read /Users/jpollock/Scripts/rsync/vdrive/NCCRC/Legal research and initiatives/non-subject-specific research/studies and reports/Houseman 2019 Civil Legal Aid Report.pdf: input/output error
2022/06/07 22:32:03 ERROR : Conferences or presentations/Master presentations and outlines/CRTC class PowerPoint.pptx: Failed to calculate dst hash: hash: failed to read: read /Users/jpollock/Scripts/rsync/vdrive/NCCRC/Conferences or presentations/Master presentations and outlines/CRTC class PowerPoint.pptx: input/output error

2

Since both the source and the destination are mounted on the filesystem, this error is coming from the filesystem.

I don't know which of the source or destination is mounted as a network filesystem, but I expect the errors are coming from there.

We've had various weird problems with rclone and smb on Linux to do with rclone tickling edge cases in the smb and kernel, but this is the first I've heard of it in macOS.

Is there some kind of log you can look at?

Does the sync eventually run clean with no errors?

3

The destination, which is a remote Windows fileserver, is mounted on the local filesystem via Apple Filing Protocol (AFP). I'm using AFP instead of SMB (which Macs can mount) because the SMB reliability was even worse: the remote mount would often disconnect in the middle of the sync, and it was terribly slow.

No, the sync never runs clean: it always has failures, but the files that fail vary from run to run.

One of the things I wondered about is whether using rclone's caching settings, like the cache time or chunk size, might help. I've read that caching is deprecated so I don't know.

I haven't access to the remote server's log, but I'll see if I can find someone who can.

4

This sounds very much like a bug, possibly in AFP but possibly in the go runtime. Go uses some of the lesser used filing system interfaces and in combination with its async preemption it can tickle kernel bugs.

You could try turning off async preemption by setting the env var GODEBUG=asyncpreemptoff=1 and see if that helps.

5

I set GODEBUG=asyncpreemptoff=1 in my .cshrc file and it seems to be helping!

6

That's good. That means it's most likely a kernel bug, or possibly a bug in go's premption. Either way, very difficult to track down.

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.