this is my first post on this forum, hope I will like it here. Did anybody already work on something like versioning for rclone? Versioning is a feature which can be easily added to any remote, but there are many ways of how it could be done, I would believe that there is already somebody who works/worked on that concept for rclone, of course having it delivered by rclone is good, if not, then I would appreciate if you can share some links to something that fits this request.
Basic tough would be:
check if file is newer than one being uploaded
if version limit is set to version amount like as example last 10 files or some size or even for a specific period, as well as all of them combined, something like this:
--versioning-limit-totalsize 100G # total max size to be kept, everything over causes oldest to be deleted
--versioning-limit-filesize 1G # do not use versioning on files bigger/equal 1G
--versioning-limit-amount 100 # max. amount of versions to be kept
--versioning-limit-time 604800 # keep only versions in range of specific time for now, useful for PVR
...
Process on versioning (first upload, then if successful delete versioning limits if reached)
That is in short how I would like to do without adding additional extras like custom naming for versioning folder or even separate path where compression also can be added, like than last 9 versions get compressed, meaning that on each version upload, 1 currently last would be additionaly compressed (as/if uncompressed).
I need it by myself and might be able to create it in code quality which is sufficient for my private use, but I think it would be amazing if we have versioning option and then one very configurable. Versioning feature is especially useful for those who work directly on mounted drives.
I am not sure what exactly do you mean by substantially better/differnt from --backup-dir? As far as I see it, --backup-dir does what it should quite well and separating by date is not really versioning concept (I would need to check, but I would preffer epoch timestamp instead of a date). In a backup you do want to ensure that all data is really backed up and valid, where with versioning you kinda do the same with additional versioning strategy which is different from backup strategies. Also from point of where data is stored it gets very intransparent and hard to manage if you use --backup-dir which misses what I will call now limits/filters. Sure, those can be achieved by scripting which is exactly why I would like to have ability for it. There are also different type of storage some, where some are great and cheap for archives (aka backups) but are terrible for versioning, where if you need both, you get then lost in overlays and flags.
Some cloud storage providers want even higher pricing for this feature which I believe easily could be implemented without breaking backward compatibility or adding a new salad of flags. Where you probably do not want to have your versioning files on same disk/storage as your live storage, which again differ in restrictions up to no restrictions if you use it locally.
Above at least is what I believe, but those who know can tell me they agree with me or help me in case that there is better way of doing it with rclone to point to it.
If people do not agree then it is a good start to find out why.
I mentioned already in first post that there are many ways how to achieve it and it is mainly by scripting, your suggestion is how I could use scripting to create something, thinnking that way one does not need rclone too as you can do everythig by scripting or writting own customizations and apps. By that one needs to drive a line what is usefull for rclone and what might not be. In current case, everything which is required for versioning is given and as you disagree with me, I say that it is out of question to use backup as versioning because it means I need further scripting which again becomes complex and is more tendy to errors and mistake which can lead to exactly the opposite of what one wants to achieve which is data loss.
There is big difference in rversionoing and backup for me. With suffix you suggest again further scripting which can be resolved in many ways including simply with bash/shell .... .
The filters can of course be used, but you do not describe how you would apply your concept for versioning of files for as example ability for collaboration, it simply involves too much scripting, not that I am against it, I do not have a problem wirtting whole applications with bash but this is exactly what I do want to prevent and make it much less complex.
As last one needs to mention that versioning is lets say part of a storage for which you anyway need backup strategy and backups, backup and versioning are clearly different things, where you can use backup concepts for realtime backups as well as realtime backups as backup replacement, many companies were wrongly consulted and ended up only proper realtime backup letting them pay ransom or backup existed but current data (versions) are gone and are not included in backups. This adds then another layer of complexity in definition.
As last, your suggestion is at least better than using more 3rd party bins and software. Like I said, backup is not new but it simply is not versioning solution.
I did not write what I do use currently as versioning is git, but like mentioned before, there are many ways how versioning can happen. Git can be used together with rclone, that is not really an issue, but rclone has in my view already everything required for versioning, thats why I suggested at all.