"Mac OS Extended (Journaled)"
That works for me as well:
etexter@seraphite ~ % rclone copy /etc/hosts TC: -vv
2021/09/07 13:38:47 DEBUG : rclone: Version "v1.56.0" starting with parameters ["rclone" "copy" "/etc/hosts" "TC:" "-vv"]
2021/09/07 13:38:47 DEBUG : Creating backend with remote "/etc/hosts"
2021/09/07 13:38:47 DEBUG : Using config file from "/Users/etexter/.config/rclone/rclone.conf"
2021/09/07 13:38:47 DEBUG : fs cache: adding new entry for parent of "/etc/hosts", "/etc"
2021/09/07 13:38:47 DEBUG : Creating backend with remote "TC:"
2021/09/07 13:38:47 DEBUG : Creating backend with remote "/Volumes/Test"
2021/09/07 13:38:47 DEBUG : hosts: Size and modification time the same (differ by 0s, within tolerance 1ns)
2021/09/07 13:38:47 DEBUG : hosts: Unchanged skipping
2021/09/07 13:38:47 INFO :
Transferred: 0 / 0 Byte, -, 0 Byte/s, ETA -
Checks: 1 / 1, 100%
Elapsed time: 0.0s
2021/09/07 13:38:47 DEBUG : 4 go routines active
etexter@seraphite ~ % diskutil list
/dev/disk0 (internal):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme 500.3 GB disk0
1: Apple_APFS_ISC 524.3 MB disk0s1
2: Apple_APFS Container disk3 494.4 GB disk0s2
3: Apple_APFS_Recovery 5.4 GB disk0s3
/dev/disk3 (synthesized):
#: TYPE NAME SIZE IDENTIFIER
0: APFS Container Scheme - +494.4 GB disk3
Physical Store disk0s2
1: APFS Volume Macintosh HD 15.3 GB disk3s1
2: APFS Snapshot com.apple.os.update-... 15.3 GB disk3s1s1
3: APFS Volume Preboot 400.7 MB disk3s2
4: APFS Volume Recovery 1.0 GB disk3s3
5: APFS Volume Data 196.5 GB disk3s5
6: APFS Volume VM 1.1 GB disk3s6
/dev/disk4 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *31.0 GB disk4
1: EFI EFI 209.7 MB disk4s1
2: Apple_HFS Test 30.7 GB disk4s2
Interesting I get this (added some debug lines to source);
./rclone copy /etc/hosts /Volumes/Blue/temp --config='' -vv
2021/09/07 19:58:37 DEBUG : rclone: Version "v1.57.0-DEV" starting with parameters ["./rclone" "copy" "/etc/hosts" "/Volumes/Blue/temp" "--config=" "-vv"]
2021/09/07 19:58:37 DEBUG : Creating backend with remote "/etc/hosts"
2021/09/07 19:58:37 DEBUG : Using config file from ""
2021/09/07 19:58:37 DEBUG : fs cache: adding new entry for parent of "/etc/hosts", "/etc"
2021/09/07 19:58:37 DEBUG : Creating backend with remote "/Volumes/Blue/temp"
window 1ns
root /etc
Created temp file /var/folders/_r/1gh5y2td155b5bf9gmk74_hh0000gn/T/rclone1336146336
Precision detected as 1ns
precision 1ns
root /Volumes/Blue/temp
Created temp file /var/folders/_r/1gh5y2td155b5bf9gmk74_hh0000gn/T/rclone1056243359
Precision detected as 1ns
precision 1ns
2021/09/07 19:58:37 DEBUG : hosts: Modification times differ by -956.462099ms: 2021-09-07 16:31:26.956462099 +0200 CEST, 2021-09-07 16:31:26 +0200 CEST
2021/09/07 19:58:37 DEBUG : hosts: md5 = e37634b906b4608c696b01434061bfd3 OK
2021/09/07 19:58:37 INFO : hosts: Updated modification time in destination
2021/09/07 19:58:37 DEBUG : hosts: Unchanged skipping
2021/09/07 19:58:37 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 1 / 1, 100%
Elapsed time: 0.0s
2021/09/07 19:58:37 DEBUG : 4 go routines active
What's your OS disk formatted as?
APFS. I added a PR to debug the detected precision here:
Encrypted as well?
Yes, exactly the same.
I can't recreate your issue with the same parameters. Not sure.
I have this with a mounted SMB share I use as a destination. I just assumed that rclone couldn't differentiate between the native resolution and the SMB mount's with crypt. I guess it is good to know it is a bug. And also you're not alone.
Personally, I just use --modify-window 1.1s to be extra sure.
This is a compromise... Rclone needs to create files to detect the precision of the file system. Creating random files in the user's destination isn't such a good idea, so rclone creates the files in the OS temporary directory.
This is correct nearly all the time, but when using external media can go wrong as you've seen.
But on a UNIX-like filesystem, a path means absolutely nothing, so that function isn't really telling us anything useful about the src nor the dst at all. We are basically checking the /tmp folder of the host and nothing else.
I think writing a file to the destination is not a problem but for the source it really is a bad idea, I agree.
I think we should default to 1 second local copy unless we can develop a read-only precision check method.
I propose to change the current readPrecision() function with the following:
Do a stat on the source or destination folder. It's read-only and very obviously tells the precision (you just count the zeros), for example:
stat -f %Fm /etc/hosts
1631025086.956462099
stat -f %Fm /Volumes/microsd/test
1623704375.000000000
It's super simple, fast and read-only operation.
The goal for rclone is to interact with cloud remote.
Changing it to 1 second could potentially break a lot of things as it's been the default for some time.
I don't think it's good idea to make those changes and using external storage is a very fringe case.
I'm only proposing to change it on local. Anyway the readPrecision() modification could solve this in a really nice way.
1 Like
I did think of this, but it isn't very reliable. If you list your file system you'll find loads of files and directories with exact second times, eg
find . -print0 | xargs -0 ls -d --full-time | grep '\.000000000'
In an ideal world there would be some kind of ioctl to call to reveal the precision of a file system...
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.