Poor performance of "rclone md5sum" on FreeBSD/TrueNAS (ZFS)?

[vicmarto]

I have created a simple test comparing the performance of "md5" vs "rclone md5sum" (v1.61.1) on FreeBSD/TrueNAS (ZFS).

1st) Create files:

dd if=/dev/zero of=/mnt/zbackup/Benchmark/19GB bs=1M count=19000
dd if=/dev/zero of=/mnt/zbackup/Benchmark/26,21GB bs=1M count=26210
dd if=/dev/zero of=/mnt/zbackup/Benchmark/21,58GB bs=1M count=21580
dd if=/dev/zero of=/mnt/zbackup/Benchmark/15,7GB bs=1M count=15700

2nd) "dd concurrent read" test:

date
dd if=/mnt/zbackup/Benchmark/19GB of=/dev/null bs=1M &
dd if=/mnt/zbackup/Benchmark/26,21GB of=/dev/null bs=1M &
dd if=/mnt/zbackup/Benchmark/21,58GB of=/dev/null bs=1M &
dd if=/mnt/zbackup/Benchmark/15,7GB of=/dev/null bs=1M &
wait
date

3th) "md5 concurrent" test:

date
md5 /mnt/zbackup/Benchmark/19GB &
md5 /mnt/zbackup/Benchmark/26,21GB &
md5 /mnt/zbackup/Benchmark/21,58GB &
md5 /mnt/zbackup/Benchmark/15,7GB &
wait
date

4th) "rclone md5sum concurrent" test:

date; /root/rclone md5sum /mnt/zbackup/Benchmark --progress; date

With these results:

According to calculations, while "md5 concurrent" is only 7.6% slower than "dd concurrent read", "rclone md5sum" is 22.2% slower.

On the other hand, repeating the same tests but on macOS 13.2 Ventura, "rclone md5sum" is only 10.1% slower than "dd concurrent read".

--
[note] The zfs dataset where the test is run is set to "compression=off".

[Ole (Ole Frost)]

Hi vicmarto,

I know your situation and it seems like your NAS has a hard disk drive (with seek times).

I therefore guess you can improve rclone performance on the NAS by increasing the --buffer-size to reduce the need for disk seeks. The default is 16M, your could try 250M if your have at least 1GB free RAM.

Knowing your situation and challenges, I suggest you try the experimental hasher. I haven't tried it, but seems like it was designed for scenarios like yours. If the hasher doesn't calculate the md5 using the NAS md5 command, then it may be a good idea to to set disable_concurrent_writes to reduce the risk of missing blocks in the file. You can do that by entering "nas,disable_concurrent_writes:" in the remote of the hasher.