is it possible to Sync Rclone and keep the files that are last modifide?
For example
example 1)
Host: Changes File A
Remote: Adds File B
in that case i would like both Files to still be there, just like GIT would do it
example 2)
Remote: has a Folder X that HOST doesnt have
Host: has some new Files/Folders Remote doesnt have
if i sync on HOST i want to download the Folder X and not override / delete it
and i want to upload all my new files to remote
sorry for my bad english so im not really sure how to formulate what i really want
i want to sync HOST & Remote just like Google Sync would do it
if i would do something like
rclone sync "S:\Google Drive\Semester 2" "hs:/Semester 2/" -P
i would override everything i changed on remote (hs:/Semester 2/) but i just want to "update" both directories
im not sure if you guys get what i mean, if not im really sorry
Let me first be sure I understand what you want to accomplish...
You basically want to keep your most recent files both on your PC and the cloud right?
In that case, it sounds like you will want to use copy instead of sync
sync will try to make location B identical to A, including overwriting anything that is different (even newer) in B. It also deletes any files in B (in that location only) that did not exist on A. After sync the two locations will be completely identical.
copy will just add everything from A to B. If file exist but has older date - it is overwritten with newest file. If date is newer then it just skips this file. copy will never delete any files (except by overwriting because of updating old files of course).
The only inconvenience here may be that if you delete files you have to delete them in both places to get rid of them.
There are some ways to modify how sync works by adding flags like --ignore-existing or --immutable (read more here): https://rclone.org/docs/#ignore-existing
But this does not sound like the best solution for you in this case.
rclone does not have a native true 2-way sync - but it should be very possible to script using the existing functions.
I hope that helps you - and if I have misunderstood what you are trying to do then please correct me
Yes, that looks fine to me. You will end up with all files in both locations - only the newest versions.
When you delete/move files you will have to delete them both places though, or else they will just be restored from the other side next time you run the script If that is really annoying - there would be ways to script around this I think, but assuming that this isn't some huge collection of files anyway then in your case it's probably not a huge deal. Or - you could manually trigger a sync after you delete/rearrange files to mirror the changes to the backup.
Be aware that if you use && then the second line will only complete if the first command was successful (ie. finished without a critical error).
If you just used 2 lines without && it should run the second line regardless.
This shouldn't matter much in practice here as nothing bad would really happen if a copy failed for some reason.
On the subject of deletion, is there a way to get a list of all the files that exist in one location but not another? I was thinking once I had that list, I could delete them individually on the other location before doing a copy.
My strategy would be to allow deletions in only in one location.
All of a sudden my office wants to mirror between Google Drive and our LAN drive because people are having to work from home because of covid-19.
--Note-- since I first posted this I realized that files added in the remote location would show up on the delete list. I think I would also have to keep track of times so that files added remotely after the sync time wouldn't be deleted.
I don't think rclone can give you a comparison-list from a built-in command. @ncw Or am I mistaken here?
What you can fairly easily do with a little scripting though is use
rclone ls (or lsl or lsf depending on what data you want in the list) https://rclone.org/commands/rclone_lsl/
And then you can script a comparison between the two easily enough.
I would think that it would be much easier to just do a sync after you delete on one side though (ideally right after you copied in both directions to make sure all files are up to date on your end). A sync will mirror any changes, including moves and deletions to the other side and make an identical copy. That seems much more straight-forward to me.
You can certainly do this in a few ways depending on the setup. Many remotes can be specified to be read-only access. Mounts can be set to be read-only on that level for end-users. The sync command can also be modified with flags like --immutable or --max-delete 0 https://rclone.org/flags/#non-backend-flags
Perhaps you would want to enable --backup-dir to keep the previous versions of modified or deleted files in case anything goes wrong (or just generally needs to be recovered - as is often needed in work situations as people make mistakes). https://rclone.org/docs/#backup-dir-dir
Normally --backup-dir only keeps one previous revision of files, but you can see here an easy example (demonstrated in batch script) of how you can make a full dated revisions system if you need that extra layer of security and administrative flexibility:
Flags:
--combined string Make a combined report of changes to this file
--differ string Report all non-matching files to this file
--dst-only string Report all files missing from the source to this file
--match string Report all matching files to this file
--src-only string Report all files missing from the destination to this file
The combined report puts a symbol in front of each file, +/-/ etc (haven't written the docs for this yet!
Thanks. Yes, this is getting me on the right track.
Just one thing. There is mention of "plugins" in the github repository. Is there plugin support for RClone? If there were hooks to tell RClone to copy or delete a file or not based on mod or creation time, that would be easier than working offline with lists. Thanks.
Fantastic Nick! (you are are on fire lately aren't you!)
This will be supremely useful for scripting and creating any custom handling you need. Not having to manually process this from raw lists simplifies things a ton
Do you suspect this will make it into mainline beta soon, or is there still some more polishing to be done?
So you can use the same flags to not only do the operation but also get this listing data?
If you can tie that into the same system I don't see why not. It could certainly be useful to for example log the changed files easily - without having to parse the main log for that info.
sync should sync - not sure how those flags would change syncing files.
copy should copy - not sure how those flags would change copying files.
move should move - not sure how those flags would change moving files.
check should check - - not sure how those flags would change checking files.
seems to me that we should have a new action verb for this functionality.
perhaps compare
??
It wouldn't change the function of these commands at all.
All it would do is allow you to dump useful comparison data from the listings it is already making to perform the operation.
If you just wanted this data without any operation happening at the same time you would use "rclone check" as Nick said. Compare is a nice verb I agree, but do we need to add that to check that already is an existing command? it might be a good command alias though as it feels more descriptive for these particular usecases even if it technically relates strongly to check under the hood. An alias wouldn't break anything (assuming it is currently unused of course).
It would be the same data that the listings return - so presumably all of the above (except maybe hashes when you are not implicitly or explicitly comparing on them).
I would assume Nick would implement it as one of the existing rclone formats so that you can input these lists into another operation in --filter-from or similar.
Something like the "rclone lsl" format I suspect.
But you really have to ask Nick for the details here. I am just making assumption based on what I think would be the most common-sense way to do it
