Was not sure where to post this in Support or Feature Section.
General question/idea regarding handling dupes
I have run into a situation where I have a ton of duplicate files based on the MD5 value.
Iterating them 1 by 1 is a bit of PITA, having to choose them 1 by 1.
In this case I am trying to keep certain files formatted in a specific manner.
So I was wondering if there was an easy way, or easy mod that would allow me to say tell rclone to move files from a folder where the MD5 value is the same.
Doing this would at least put all files with matching MD5 in the same folder which would actually make it easier and faster to visually look at the files and choose the ones to delete etc
Is there current way of moving dupes based on MD5 I am not aware of?
I would use rclone dedupe -dedupe-mode list to generate the list of all duplicates and then parsed results with simple bash script applying whatever action is required (e.g. move).
never thought of the list mode, but bash script would be issue for me since I am windows geek and not familiar with writing bash, but I will give it a try using EmEditor instead.
Does not have to be bash. Can be some windows scripting stuff.
As you want dedupe based on hash also add --by-hash.
It will work like this:
$ rclone dedupe --dedupe-mode list --by-hash drive:dups
2024/07/27 20:24:41 NOTICE: 654abf312e8c1e514eea68f42ccaedec: Found 2 files with duplicate md5 hashes
654abf312e8c1e514eea68f42ccaedec: 2 duplicates
1: 6827 bytes, 2013-10-01 09:32:42.500000000, IL 00OCTa.xlsx
2: 6827 bytes, 2013-10-01 09:32:42.500000000, IL 00OCT.xlsx
Output does not look very scripting friendly:( so might be some scripting voodoo needed. But on the other hand if you do not have too many dupes you can manually (with maybe some help of grep etc.) edit it and create list of files to be moved and then rclone move --files-from my_list
This is at least how I would approach it. Maybe somebody has better idea.