i have the problem when i download from my mounted drive via ftp that has many file checksum mismatch. so i can’t unpack my rar file anymore after the download.
on the server where rclone directly mounted the dropbox i can also extract the files and the crc value is the same. it comes to me soor as if i had the files as duplicate on my dropbox. once good and once damaged
they are mounted on the server via rclone. if i download them over the ftp the files are not all broken but at least 30 percent there the crc value is wrong.
but when i copy it to the server where rclone is mounted via cp down to the locale hdd of the server the data is whole then the crc value is correct again and the files can be unpacked.
if i check on the rclone mounted drive (server) the crc value is it correct if i load the files home outside from the server they are broken.
i can also extract the file from the rclone drive as example.
command:
cd /local/hdd/drive
rar e /opt/rclone/mount/myrarfile.rar to local drive from the server all files OK and i can extract without a error.
i checked the server for heart and kidneys. the hdds are ok the ram also
#########
it looks like unionsfs makes the trouble if i download via direct mount from rclone works fine but if i download over unionsfs mount they are a lot of files broken. gives any experiences with it ?
thanks for the fast answers but i think calisro have right.
my system have mostly some load about encfs and rclone sync/copy about big files.
i have tested now the files are all good without crc error. without any changes now have my server little bit less load than yesterday very confusing.
i have tested too Animosity002 settings is like the same as with the options. but i cant now Replicate the error
how good and stable (fast like unionsfs) works mergerfs ? i have no experience about this overlayfs i have seen it gives a fork from unionsfs called “aufs”
#########
@Animosity022 is it possible in mergersfs works similar with read only and write flags as unionfs?
like this:
I would try removing options from the union-fs command line and see if any of those affect the problem. So try removing -o allow_other -o cow -o direct_io -o auto_cache -o sync_read and see what happens.
You are doing a rclone copy from a remote to local and trying to unpack it and it works on the rclone mount or local mount, but does not work on the unionfs or mergerfs mount.
Is that the right flow? Can you share specific commands from start to finish and I can try to test.
i have tested both mounts like unionfs and mergerfs i have now permanently the crc problem with some files so i can replicate.
###########
download direct from rclone mountpoint /opt/drop2 i have no problems it works perfect without any errors.
like cd /tmp ; rar e /opt/drop2/mydirectory/filename.rar works perfect same for download.
example:
works without error cp /opt/drop2/mydirectory/filename.rar /tmp
works not crc error cp /opt/allinone/mydirectory/filename.rar /tmp
if i try to download over unionfs/mergerfs i have trouble with some files (always the same files) download tested via winscp, ftpserver, xftp client still the error occurs with the checksum.
you have a message with the download link about the files good.r14 and bad.r14 both files are the same just the difference is good.r14 is downloaded directly from rclone mount and the bad.r14 is downloaded via unionsfs
What I learnt from studying them is that the corruptions come in 4k chunks and on 4k boundaries. They appear to be random data or data from a different file. There doesn’t seem to be any other pattern.
The fact that they are in 4k chunks points more towards a kernel problem as 4k is the page size the kernel will be using.
What size chunks does rclone read with? I looked back at the logs you sent but they don’t actually show a read operation.