I'm on Windows and I use Rclone to sync my data between two local ssd
If my folder are named for example hello and I rename to Hello or HELLO only the content of the folder
are sync but the folder name stay hello instead Hello
And if I have a file Document.txt, if I only rename the document to DOCUMENT.TXT without editing content, the new filename are not sync
How can I force to sync the new folder name if I change lowercase or uppercase in the folder name ?
I have try to sync with the arguments --track-renames, --modify-window and --checksum without success
Thank you for your help !
Rclone version: rclone v1.57.0
If I run a simple test with -vv:
rclone sync -vv ./src ./dest
The result is
DEBUG : rclone: Version "v1.57.0" starting with parameters ["C:\\rclone.exe" "sync" "-vv" "./src" "./dest"]
DEBUG : Creating backend with remote "./src"
DEBUG : Using config file from "C:\\Users\\xxx\\.config\\rclone\\rclone.conf"
DEBUG : fs cache: renaming cache item "./src" to be canonical "//?/C:/src"
DEBUG : Creating backend with remote "./dest"
DEBUG : fs cache: renaming cache item "./dest" to be canonical "//?/C:/dest"
DEBUG : Local file system at //?/C:/dest: Waiting for checks to finish
DEBUG : HELLO/DOCUMENT.TXT: Size and modification time the same (differ by 0s, within tolerance 100ns)
DEBUG : HELLO/DOCUMENT.TXT: Unchanged skipping
DEBUG : Local file system at //?/C:/dest: Waiting for transfers to finish
DEBUG : Waiting for deletions to finish
INFO : There was nothing to transfer
INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 1 / 1, 100%
Elapsed time: 0.4s
DEBUG : 2 go routines active
Rclone won't rename the file as it thinks FILE and file are the same thing on Windows and Dropbox.
You can try using the flag --disable CaseInsensitive - you probably want to use this with --track-renames otherwise you'll upload all the mismatched files rather than rename them.
Definitely do that! I ran some quick tests, and it doesn't work great, in some cases ended up just deleting the destination (tested copy and sync, with and without --track-renames, so I don't remember exactly what was better or worse)
Using --delete-before works without errors, but the downside, at least if the destination is a cloud storage, is that it will delete (the existing with wrong casing) and then make a new copy (from source with correct casing):
Thank you for your answer ! I did some research on the case sensitive on the official
documentation I saw:
Normally the local backend declares itself as case insensitive on Windows/macOS and case sensitive for everything else. Use this flag to override the default choice.
Is there a reason why it shouldn't be case sensitive by default on Windows/MacOS ?
Aha, so there are dedicated options --local-case-sensitive and --local-case-insensitive on the local backend. Yes: These are used to turn on/off the CaseInsensitive feature, so you can call it an alias.
I misread the original post and thought you were syncing to dropbox for some reason! Dropbox is also case insensitive and --local-case-sensitive wouldn't work for that, but it is the perfect solution for local -> local copies.
Thanks for testing.
This is expected
But this isn't! What error does it give?
I think a better long term solution for this would be to have an analogue of this flag
--ignore-case-sync Ignore case when synchronizing
Maybe
--no-ignore-case-sync Don't ignore case when synchronizing
I suspect there are some side effects of --disable CaseInsensitive that I haven't thought of!
2022/01/20 17:11:01 DEBUG : Creating backend with remote "C:\\Temp\\x"
2022/01/20 17:11:03 DEBUG : Reset feature "CaseInsensitive"
2022/01/20 17:11:03 DEBUG : fs cache: renaming cache item "C:\\Temp\\x" to be canonical "//?/C:/Temp/x"
2022/01/20 17:11:03 DEBUG : Creating backend with remote "C:\\Temp\\y"
2022/01/20 17:11:03 DEBUG : Reset feature "CaseInsensitive"
2022/01/20 17:11:03 DEBUG : fs cache: renaming cache item "C:\\Temp\\y" to be canonical "//?/C:/Temp/y"
2022/01/20 17:11:03 INFO : Local file system at //?/C:/Temp/y: Making map for --track-renames
2022/01/20 17:11:03 INFO : Local file system at //?/C:/Temp/y: Finished making map for --track-renames
2022/01/20 17:11:03 DEBUG : Local file system at //?/C:/Temp/y: Waiting for checks to finish
2022/01/20 17:11:03 DEBUG : Local file system at //?/C:/Temp/y: Waiting for renames to finish
2022/01/20 17:11:03 INFO : test.txt: Deleted
2022/01/20 17:11:03 ERROR : TEST.txt: Couldn't move: rename \\?\C:\Temp\y\TEST.txt \\?\C:\Temp\y\test.txt: The system cannot find the file specified.
2022/01/20 17:11:03 DEBUG : test.txt: Failed to rename to "TEST.txt": rename \\?\C:\Temp\y\TEST.txt \\?\C:\Temp\y\test.txt: The system cannot find the file specified.
2022/01/20 17:11:03 DEBUG : Local file system at //?/C:/Temp/y: Waiting for transfers to finish
2022/01/20 17:11:03 DEBUG : test.txt: md5 = 7897c968e70853d4c010329863f9dfe0 OK
2022/01/20 17:11:03 INFO : test.txt: Copied (new)
2022/01/20 17:11:03 ERROR : Local file system at //?/C:/Temp/y: not deleting files as there were IO errors
2022/01/20 17:11:03 ERROR : Local file system at //?/C:/Temp/y: not deleting directories as there were IO errors
2022/01/20 17:11:03 ERROR : Attempt 1/3 failed with 2 errors and: not deleting files as there were IO errors
2022/01/20 17:11:03 DEBUG : test.txt: Size and modification time the same (differ by 0s, within tolerance 100ns)
2022/01/20 17:11:03 DEBUG : test.txt: Unchanged skipping
2022/01/20 17:11:03 INFO : Local file system at //?/C:/Temp/y: Making map for --track-renames
2022/01/20 17:11:03 INFO : Local file system at //?/C:/Temp/y: Finished making map for --track-renames
2022/01/20 17:11:03 DEBUG : Local file system at //?/C:/Temp/y: Waiting for checks to finish
2022/01/20 17:11:03 DEBUG : Local file system at //?/C:/Temp/y: Waiting for renames to finish
2022/01/20 17:11:03 DEBUG : Local file system at //?/C:/Temp/y: Waiting for transfers to finish
2022/01/20 17:11:03 DEBUG : Waiting for deletions to finish
2022/01/20 17:11:03 ERROR : Attempt 2/3 succeeded
2022/01/20 17:11:03 INFO :
Transferred: 1.913 KiB / 1.913 KiB, 100%, 0 B/s, ETA -
Checks: 4 / 4, 100%
Deleted: 1 (files), 0 (dirs)
Transferred: 1 / 1, 100%
Elapsed time: 2.5s
2022/01/20 17:11:03 DEBUG : 3 go routines active
Hmm, not quite sure what is happening there but it is something to do with the fact that rclone things TEST.txt and test.txt are different files with --disable CaseInsensitive, but they aren't really.
Not sure either, but think it traces back to the os.rename, and the Windows API call it resolves to:
MoveFileEx(from, to, MOVEFILE_REPLACE_EXISTING)
Probably that flag MOVEFILE_REPLACE_EXISTING makes it trip when source and destination are the same file on disk:
If a file named lpNewFileName exists, the function replaces its contents with the contents of the lpExistingFileName file, provided that security requirements regarding access control lists (ACLs) are met. For more information, see the Remarks section of this topic.
hi, this is very interesting and complicated, but in my testing, there were no deletions?
also, i am confused about the log from below, i am missing something important.
what exactly is being server-side moved; what is the source file and what is the dest file?
why the need for server-side move, as rclone should just rename the file and be done with it?
thanks
rclone lsl b:\y
1 2022-01-20 13:57:20.014000000 TEXT.TXT
rclone sync b:\x b:\y -vv --track-renames --disable=CaseInsensitive
DEBUG : Setting --config "C:\\data\\rclone\\scripts\\rclone.conf" from environment variable RCLONE_CONFIG="C:\\data\\rclone\\scripts\\rclone.conf"
DEBUG : rclone: Version "v1.57.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "sync" "b:\\x" "b:\\y" "-vv" "--track-renames" "--disable=CaseInsensitive"]
DEBUG : Creating backend with remote "b:\\x"
DEBUG : Using config file from "C:\\data\\rclone\\scripts\\rclone.conf"
DEBUG : Reset feature "CaseInsensitive"
DEBUG : fs cache: renaming cache item "b:\\x" to be canonical "//?/b:/x"
DEBUG : Creating backend with remote "b:\\y"
DEBUG : Reset feature "CaseInsensitive"
DEBUG : fs cache: renaming cache item "b:\\y" to be canonical "//?/b:/y"
INFO : Local file system at //?/b:/y: Making map for --track-renames
INFO : Local file system at //?/b:/y: Finished making map for --track-renames
DEBUG : Local file system at //?/b:/y: Waiting for checks to finish
DEBUG : Local file system at //?/b:/y: Waiting for renames to finish
INFO : TEXT.TXT: Moved (server-side) to: test.txt
INFO : test.txt: Renamed from "TEXT.TXT"
DEBUG : Local file system at //?/b:/y: Waiting for transfers to finish
DEBUG : Waiting for deletions to finish
INFO : There was nothing to transfer
INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 2 / 2, 100%
Renamed: 1
Elapsed time: 0.0s
rclone lsl b:\y
1 2022-01-20 13:57:20.014000000 test.txt
Strange. Your log matches the more normal case, when there is a rename that is not (just) casing, and --disable CaseInsensitive does not make a difference:
C:\Temp\x\test2.txt
C:\Temp\y\test.txt
rclone sync C:\Temp\x C:\Temp\y --track-renames -vv
...
2022/01/20 22:44:25 INFO : test.txt: Moved (server-side) to: test2.txt
2022/01/20 22:44:25 INFO : test2.txt: Renamed from "test.txt"
...
The server-side operation is in reality the rename operation, so it is done on the destination. The rename operation calls the regular server-side move function internally, and it does its own logging:
test.txt: Moved (server-side) to: test2.txt
Then, when finished, the rename operation ends with its final status:
test2.txt: Renamed from "test.txt"
So, to conclude: Both of these log messages are from the same operation - a server-side move done as part of a rename due to option --track-rename.
