I'm trying to use rclone to run a scheduled sync to a nas, and it takes a long time because it always checks the checksum if the size and time don't match. I found --size-only works to limit it to only the size check, but I would also like it to check the time, but not the checksum. I haven't been able to find a flag for this.
rclone, by default, checks size and time.
Doesn’t transfer unchanged files, testing by size and modification time
However I've found that without --size-only the speed slows to an absolute crawl. This is my command
/usr/bin/rclone sync --verbose --transfers 30 --stats 1s --size-only "/home/ling" "/mnt/pi/laptop"'
Without --size-only it says "checking" on every small file, and transfer speeds are around 5 kb/s.
My nas is mounted via cifs-utils
on which device are you running rclone?
I'm running it on my laptop
when you posted, you should have seen a template, asking questions for info...
Ah on Ubuntu 20
can you supply the requested info...
What is the problem you are having with rclone?
Transfer is extremely slow without the --size-only option, but then it doesn't check time
What is your rclone version (output from rclone version)
rclone v1.50.2
- os/arch: linux/amd64
- go version: go1.13.6
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Ubuntu 20 64 bit
Which cloud storage system are you using? (eg Google Drive)
Mounted NAS (mounted on /mnt/pi)
The command you were trying to run (eg rclone copy /tmp remote:tmp)
/usr/bin/rclone sync --verbose --transfers 30 --stats 1s --size-only "/home/ling" "/mnt/pi/laptop"'
The rclone config contents with secrets removed.
A log from the command with the -vv flag
you are using an old version of rclone.
to update
curl https://rclone.org/install.sh | sudo bash
I have updated rclone, but the problem persists. Without --size-only it is now on the order of bytes per second. On all files "checking" takes a long time, and it only rarely finds an updated file to sync.
I have made a revelation! I recreated a configuration for my NAS using SFTP instead, and now it transfers fine without the --size-only flag. However I'm still curious as to how speeds were so low when using cifs-utils to mount the samba share.
glad you got it working
Speeds were normal with cifs with the --size-only flag, unless reading modification time has a huge latency which I haven't experienced I'm not sure how this would be affected
If the log you shared was when it was slow, it was checking MD5 sums:
2020/05/17 13:34:02 DEBUG : .config/pavucontrol.ini: MD5 = 13d0e3be3518430accd7102fa3255891 OK
2020/05/17 13:34:02 DEBUG : .cache/motd.legal-displayed: MD5 = d41d8cd98f00b204e9800998ecf8427e OK
2020/05/17 13:34:02 DEBUG : .config/user-dirs.dirs: MD5 = e96ec7012d30be266ba6e6d81e88228d OK
I'd guess SFTP is working because it can't find a md5sum command on the other side. The debug would confirm that.
Yeah, that was a log from when it was slow, not sure why it was checking sums then when it wasn't specified, is there a way to explicitly disable that or a workaround for cifs?
Speeds are still slow with that flag, it might be an issue with cifs, but at least sftp works fine.
You'd have to share a debug log.
Here is a log of the command
/usr/bin/rclone sync --verbose --transfers 30 --stats 1s --ignore-checksum -vv "/home/ling" "/mnt/pi/laptop"