Attn: @ncw , you probably want to have a look at this
Hey all,
Recently started noticing dedupe looked like it was getting stuck for no reason and not even giving error output with -v . After some troubleshooting I figured out it was a problem with using -P (progress indicator).
What seems to happen is that when dedupe (in interactive mode) pauses to ask the user for input -P will overwrite most of the question and "menu", thus basically breaking it and halting indefinitely for no apparent reason.
It looks something like this:
C:\rclone>rclone dedupe TD1: -vv --tpslimit 5 -P
2019/08/01 16:46:44 DEBUG : rclone: Version "v1.48.0-082-ga35aa136-beta" starting with parameters ["rclone" "dedupe" "TD1:" "-vv" "--tpslimit" "5" "-P"]
2019/08/01 16:46:44 DEBUG : Using config file from "C:\rclone\rclone.conf"
2019/08/01 16:46:44 INFO : Starting HTTP transaction limiter: max 5 transactions/s with burst 1
2019-08-01 16:46:44 INFO : Google drive root '': Looking for duplicates using interactive mode.
2019-08-01 16:47:03 NOTICE: Crypt1/nlc28rgi1rdqf3mf8b7h5ut314a5kejs41j9usivvcdv0iib4rb0: Found 2 duplicates - deleting identical copies
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 0, -
Elapsed time: 18.5sCrypt1/nlc28rgi1rdqf3mf8b7h5ut314a5kejs41j9usivvcdv0iib4rb0: 2 duplicates remain
1: 2376763893 bytes, 2019-07-31 15:02:11.802000000, MD5 cfbd654ba0d868ede3d6389ad13aa4c5
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 0, -
Note that I am on Windows10, so it's not impossible that Linux output is slightly different and would not have an identical problem. It would be really nice if someone on Linux could test and see if they see the same thing there (you probably will need to have an actual duplicate to clean up to see the issue though).
If the same problem happens on Linux then the solution is probably to just ignore -P when interactive mode is used on dedupe.
If this is a windows specific issue then I'm not sure what the best fix is, except to maybe look into if -P can be modified to work better on this OS. If not (or if it's just not worth the time to do so) then I assume it's probably possible ignore -P with dedupe in interactive mode only for the windows version.
Ultimately this is not such a big problem since it's easy to avoid the whole issue, but it is a bit of a user-trap to allow this to happen as it will cause a lot of confusion and necessary frustration for less technical users.
I will create an issue for this once i have confirmation regarding how it looks on Linux.
