I'm trying to sync from a FreeBSD machine (zfs filesystem case sensitive) to a macOS machine (APFS filesystem case insensitive ). The problems is with file cases.
For example, if I do in the FreeNAS machine: mv /mnt/P/file /mnt/P/File, and then sync with rclone, the /Volumes/HD/P/file don't renames to /Volumes/HD/P/File.
Tried too with --local-case-insensitive and --local-case-sensitive with no luck. How can rclone be forced to respect case? Thanks!
What is your rclone version (output from rclone version)
rclone v1.53.3-DEV
os/arch: darwin/amd64
go version: go1.15.5
Which OS you are using and how many bits (eg Windows 7, 64 bit)
macOS Catalina
Which cloud storage system are you using? (eg Google Drive)
SFTP
The command you were trying to run (eg rclone copy /tmp remote:tmp)
2020/12/02 22:38:46 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "sync" "nas:/mnt/P" "/Volumes/HD/P" "--track-renames" "-P" "--fast-list" "-vv"]
2020/12/02 22:38:46 DEBUG : Using config file from "/Users/vicmarto/.config/rclone/rclone.conf"
2020/12/02 22:38:46 DEBUG : Creating backend with remote "nas:/mnt/P"
2020/12/02 22:38:46 DEBUG : sftp://vicmarto@10.0.0.250:22//mnt/P: New connection 10.0.0.100:50043->10.0.0.250:22 to "SSH-2.0-OpenSSH_8.0-hpn14v15"
2020/12/02 22:38:46 DEBUG : Creating backend with remote "/Volumes/HD/P"
2020-12-02 22:38:46 DEBUG : File: Size and modification time the same (differ by 0s, within tolerance 1s)
2020-12-02 22:38:46 DEBUG : File: Unchanged skipping
2020-12-02 22:38:46 INFO : Local file system at /Volumes/HD/P: Making map for --track-renames
2020-12-02 22:38:46 INFO : Local file system at /Volumes/HD/P: Finished making map for --track-renames
2020-12-02 22:38:46 DEBUG : Local file system at /Volumes/HD/P: Waiting for checks to finish
2020-12-02 22:38:46 DEBUG : Local file system at /Volumes/HD/P: Waiting for renames to finish
2020-12-02 22:38:46 DEBUG : Local file system at /Volumes/HD/P: Waiting for transfers to finish
2020-12-02 22:38:46 DEBUG : Waiting for deletions to finish
2020-12-02 22:38:46 INFO : There was nothing to transfer
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 1 / 1, 100%
Elapsed time: 0.0s
2020/12/02 22:38:46 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 1 / 1, 100%
Elapsed time: 0.0s
2020/12/02 22:38:46 DEBUG : 14 go routines active
Thanks ncw for your answer, and congratulations for such a great program, so useful and flexible. I really appreciate the --track-renames switch.
--local-case-sensitive "works", the file gets renamed correctly, but rclone outputs some errors:
2020/12/03 08:42:49 DEBUG : Using config file from "/Users/vicmarto/.config/rclone/rclone.conf"
2020/12/03 08:42:49 DEBUG : Creating backend with remote "nas:/mnt/P"
2020/12/03 08:42:49 DEBUG : sftp://vicmarto@10.0.0.250:22//mnt/P: New connection 10.0.0.100:50305->10.0.0.250:22 to "SSH-2.0-OpenSSH_8.0-hpn14v15"
2020/12/03 08:42:49 DEBUG : Creating backend with remote "/Volumes/HD/P"
2020-12-03 08:42:49 INFO : Local file system at /Volumes/HD/P: Making map for --track-renames
2020-12-03 08:42:49 INFO : Local file system at /Volumes/HD/P: Finished making map for --track-renames
2020-12-03 08:42:49 DEBUG : Local file system at /Volumes/HD/P: Waiting for checks to finish
2020-12-03 08:42:49 DEBUG : Local file system at /Volumes/HD/P: Waiting for renames to finish
2020-12-03 08:42:49 DEBUG : sftp cmd = /mnt/P/File
2020-12-03 08:42:49 DEBUG : sftp output = "d41d8cd98f00b204e9800998ecf8427e /mnt/P/File\n"
2020-12-03 08:42:49 DEBUG : sftp hash = "d41d8cd98f00b204e9800998ecf8427e"
2020-12-03 08:42:49 INFO : File: Deleted
2020-12-03 08:42:49 ERROR : file: Couldn't move: rename /Volumes/HD/P/file /Volumes/HD/P/File: no such file or directory
2020-12-03 08:42:49 DEBUG : File: Failed to rename to "file": rename /Volumes/HD/P/file /Volumes/HD/P/File: no such file or directory
2020-12-03 08:42:49 DEBUG : Local file system at /Volumes/HD/P: Waiting for transfers to finish
2020-12-03 08:42:49 DEBUG : File: MD5 = d41d8cd98f00b204e9800998ecf8427e OK
2020-12-03 08:42:49 INFO : File: Copied (new)
2020-12-03 08:42:49 ERROR : Local file system at /Volumes/HD/P: not deleting files as there were IO errors
2020-12-03 08:42:49 ERROR : Local file system at /Volumes/HD/P: not deleting directories as there were IO errors
2020-12-03 08:42:49 ERROR : Attempt 1/3 failed with 2 errors and: not deleting files as there were IO errors
2020-12-03 08:42:49 INFO : Local file system at /Volumes/HD/P: Making map for --track-renames
2020-12-03 08:42:49 DEBUG : File: Size and modification time the same (differ by 0s, within tolerance 1s)
2020-12-03 08:42:49 DEBUG : File: Unchanged skipping
2020-12-03 08:42:49 INFO : Local file system at /Volumes/HD/P: Finished making map for --track-renames
2020-12-03 08:42:49 DEBUG : Local file system at /Volumes/HD/P: Waiting for checks to finish
2020-12-03 08:42:49 DEBUG : Local file system at /Volumes/HD/P: Waiting for renames to finish
2020-12-03 08:42:49 DEBUG : Local file system at /Volumes/HD/P: Waiting for transfers to finish
2020-12-03 08:42:49 DEBUG : Waiting for deletions to finish
2020-12-03 08:42:49 ERROR : Attempt 2/3 succeeded
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 4 / 4, 100%
Deleted: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.0s
2020/12/03 08:42:49 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 4 / 4, 100%
Deleted: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.0s
2020/12/03 08:42:49 DEBUG : 14 go routines active```
The problem with the server side rename is here, because we've told rclone that the destination is case insensitive, then it things that "File" and "file" are two different files, so before doing the rename it deletes "File" to get it out the way. When it comes to do the rename it finds that "file" is now missing because the file system is actually case sensitive
If you do the sync without --track-renames but with --local-case-sensitive then it will work properly I think. However then you'll be transferring stuff you don't need to.
I tried your suggestion to not use --track-renames, unfortunately with no luck, sorry. After rename file to File in the server, seems like file was deleted locally, but File was not transferred.
Here is the log:
2020/12/04 06:34:01 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "sync" "nas:/mnt/P" "/Volumes/HD/P" "-P" "--fast-list" "-vv" "--local-case-sensitive"]
2020/12/04 06:34:01 DEBUG : Using config file from "/Users/vicmarto/.config/rclone/rclone.conf"
2020/12/04 06:34:01 DEBUG : Creating backend with remote "nas:/mnt/P"
2020/12/04 06:34:01 DEBUG : sftp://vicmarto@10.0.0.250:22//mnt/P: New connection 10.0.0.100:54441->10.0.0.250:22 to "SSH-2.0-OpenSSH_8.0-hpn14v15"
2020/12/04 06:34:01 DEBUG : Creating backend with remote "/Volumes/HD/P"
2020-12-04 06:34:01 DEBUG : Local file system at /Volumes/HD/P: Waiting for checks to finish
2020-12-04 06:34:01 DEBUG : Local file system at /Volumes/HD/P: Waiting for transfers to finish
2020-12-04 06:34:01 DEBUG : sftp cmd = /mnt/P/File
2020-12-04 06:34:01 DEBUG : sftp output = "d41d8cd98f00b204e9800998ecf8427e /mnt/P/File\n"
2020-12-04 06:34:01 DEBUG : sftp hash = "d41d8cd98f00b204e9800998ecf8427e"
2020-12-04 06:34:01 DEBUG : File: MD5 = d41d8cd98f00b204e9800998ecf8427e OK
2020-12-04 06:34:01 INFO : File: Copied (new)
2020-12-04 06:34:01 DEBUG : Waiting for deletions to finish
2020-12-04 06:34:01 INFO : file: Deleted
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 1 / 1, 100%
Deleted: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.0s
2020/12/04 06:34:01 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 1 / 1, 100%
Deleted: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.0s````
I see, it transferred "File" then deleted "file" which of course was the same object
So I think we've come to a dead end with --local-case-sensitive
I suspect this might need another flag like --fix-case which specifically looks for files on case sensitive file systems that are in the wrong case and renames them.
