Zfs bug with vfs cache

There is a known bug with the default zfs from ubuntu 22.04, and since updating it is not trivial at the moment, and probably lots of people are using zfs either at the root, cache directory etc...

I would like to know what rclone would do if it suffered this bug?

Also it appears the bug is more likely to appear if you move/write files in a certain way, is it know if rclone behavior.

Also it appears the bug is related to hole/sparse files If I understand right

So while I would never say that no one has ever hit the problem unknowingly, I feel pretty confident that they haven't. And if you're not sure, ask yourself if you've ever had highly parallel workloads that involve writing and seeking the same files at the same moment.

from: Topicbox

Rclone is always writing and seeking the same file simultaneously and in parallel too right?

It is not a bug in rclone:)

Now what rclone does when filesystem serves corrupted data? Nothing. It trusts that filesystem is right. Like any other program.

And in terms of this particular ZFS issue - it has been fixed since 01/DEC/2023:

This release contains an important fix for a data corruption bug. Full details are in the issue (#15526) and bug fix (#15571). There's also a developer's bug summary that gives a good overview. We recommend everyone either upgrade to 2.2.2 or 2.1.14 to get the fix. The bug can cause data corruption due to an incorrect dirty dnode check. This bug is very hard to hit, and really only came to light due to changes in cp in coreutils 9.x.

ZFS in Ubuntu I think is still experimental so I assume that people using it know what they are doing and updated their zfsutils already.

zfs is not experimental lmao.

It's very used and no, most people won't know about a obscure silent bug discovered recently that there isn't even official patches yet unless you compile it yourself.

I want to know from @ncw how rclone would behave if this bug happened, if it's likely to happen due to how rclone works = lots of parallel reads, sparse files, always rewriting data etc

right now i have files with 324 parallel reads, and probably can achieve even 1000+ simultaneous reads of the same file

and maybe it's just a coincidence, but i'm also having weird issues happening in a lot of servers recently and i still don't know the cause, nothing in the logs and i have no reason to think lots of nvme disks are just having worse performance randomly.

I would prefer to also only update zfs if ncw think it's really required

ZFS on root is still experimental afaik... (but I agree it's pretty semantic at this point)

I don't use zfs at root, just at the rclone cache directory

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.