Trouble with CRC

i have the problem when i download from my mounted drive via ftp that has many file checksum mismatch. so i can’t unpack my rar file anymore after the download.

on the server where rclone directly mounted the dropbox i can also extract the files and the crc value is the same. it comes to me soor as if i had the files as duplicate on my dropbox. once good and once damaged

how exactly can I check that?

Are you saying the file is OK when you download it on the server? Or bad?

they are mounted on the server via rclone. if i download them over the ftp the files are not all broken but at least 30 percent there the crc value is wrong.

but when i copy it to the server where rclone is mounted via cp down to the locale hdd of the server the data is whole then the crc value is correct again and the files can be unpacked.

if i check on the rclone mounted drive (server) the crc value is it correct if i load the files home outside from the server they are broken.

i can also extract the file from the rclone drive as example.
command:
cd /local/hdd/drive
rar e /opt/rclone/mount/myrarfile.rar to local drive from the server all files OK and i can extract without a error.

i checked the server for heart and kidneys. the hdds are ok the ram also

#########

it looks like unionsfs makes the trouble if i download via direct mount from rclone works fine but if i download over unionsfs mount they are a lot of files broken. gives any experiences with it ?

unionfs-fuse -o allow_other -o cow -o direct_io -o auto_cache -o sync_read /opt/incoming=RW:/opt/drop1=RO:/opt/drop2=RO /opt/allinone

Hmm, interesting…

Were there any ERRORs in the rclone log while you were downloading the corrupted file via unionfs?

@Animosity022 I think you use unionfs don’t you?

I actually use mergerfs as I didn’t like the .hidden stuff on unionfs and always found it to be problematic for me.

You could try to remove some lines from the unionfs and simplify it a bit as you only need:

unionfs-fuse -o allow_other  -o sync_read 

The allow other and sync_read.

I’ve seen unionfs corrupt files before especially during CPU changes (I was using cpulimit). Might be a bug.

thanks for the fast answers but i think calisro have right.

my system have mostly some load about encfs and rclone sync/copy about big files.

i have tested now the files are all good without crc error. without any changes now have my server little bit less load than yesterday very confusing.

i have tested too Animosity002 settings is like the same as with the options. but i cant now Replicate the error :frowning:

how good and stable (fast like unionsfs) works mergerfs ? i have no experience about this overlayfs i have seen it gives a fork from unionsfs called “aufs”

#########

@Animosity022 is it possible in mergersfs works similar with read only and write flags as unionfs?
like this:

unionfs-fuse -o allow_other -o cow -o direct_io -o auto_cache -o sync_read /opt/incoming=RW:/opt/drop1=RO:/opt/drop2=RO /opt/allinone

path1: /opt/incoming with write flags
path2. /opt/drop1 with read only
path3: /opt/drop2 with read only
mounted path: /opt/allinone

problems are back with unionsfs i have tested mergerfs it is similar the problem with crc is here too it looks like maybe a fuse bug?

mergerfs -o minfreespace=4G,defaults,allow_other,category.create=ff /opt/incoming=RW:/opt/drop1=RO:/opt/drop2=RO /opt/allinone

the direct mounts from rclone works fine without any crc errors is very strange i have send @ncw my log file i hope maybe can bring light in the dark :frowning:

I couldn’t see anything obvious in the logs…

I would try removing options from the union-fs command line and see if any of those affect the problem. So try removing -o allow_other -o cow -o direct_io -o auto_cache -o sync_read and see what happens.

So what’s your actual work flow?

You are doing a rclone copy from a remote to local and trying to unpack it and it works on the rclone mount or local mount, but does not work on the unionfs or mergerfs mount.

Is that the right flow? Can you share specific commands from start to finish and I can try to test.

@ncw very strange :frowning:

@Animosity022
rclone mount --read-only --allow-other --buffer-size 1G --timeout 5s --contimeout 5s drop1:/ /opt/drop1 &
rclone mount --read-only --allow-other --buffer-size 1G --timeout 5s --contimeout 5s drop2:/ /opt/drop2 &

unionfs-fuse /opt/incoming=RW:/opt/drop1=RO:/opt/drop2=RO /opt/allinone/

or

mergerfs -o minfreespace=4G,defaults,category.create=ff /opt/incoming=RW:/opt/drop1=RO:/opt/drop2=RO /opt/allinone

i have tested both mounts like unionfs and mergerfs i have now permanently the crc problem with some files so i can replicate.

###########

download direct from rclone mountpoint /opt/drop2 i have no problems it works perfect without any errors.
like cd /tmp ; rar e /opt/drop2/mydirectory/filename.rar works perfect same for download.

example:

works without error cp /opt/drop2/mydirectory/filename.rar /tmp
works not crc error cp /opt/allinone/mydirectory/filename.rar /tmp

if i try to download over unionfs/mergerfs i have trouble with some files (always the same files) download tested via winscp, ftpserver, xftp client still the error occurs with the checksum.

Is there anything special about these files?

Are they the correct length when you download them?

Could you put an OK file, plus one with a CRC error somewhere so I can examine them?

you have a message with the download link about the files good.r14 and bad.r14 both files are the same just the difference is good.r14 is downloaded directly from rclone mount and the bad.r14 is downloaded via unionsfs

Thanks for the files…

What I learnt from studying them is that the corruptions come in 4k chunks and on 4k boundaries. They appear to be random data or data from a different file. There doesn’t seem to be any other pattern.

The fact that they are in 4k chunks points more towards a kernel problem as 4k is the page size the kernel will be using.

What size chunks does rclone read with? I looked back at the logs you sent but they don’t actually show a read operation.

i use the standard settings that are set by the system/rclone is there a way to read it so I can share it with you?

Linux debian 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux
debian_version 9.6
rclone v1.45

  • os/arch: linux/amd64
  • go version: go1.11.2

Can you share me another log of the problem with -vv please?