I’m using rclone to migrate between different custom S3 clusters, and I have often experimented with a lot of options to optimize speed. However, it’s often difficult to know wether a spesific option has any effect at all when using the different main commands. F.ex. there is no mention of –fast-list beeing effective when using copy from a local file dump with --files-from option, although it might be logical for some. Using –color always has no effect on other than the tree command (maybe?). Another example is that I removed these 3 options:
--use-server-modtime --no-update-modtime --buffer-size 128M --s3-no-system-metadata
And measurements showed that removal of them had no effect on the copy speed at all. So what I’m suggesting is maybe some feedback under verbose option when combining options that doesn’t make any sense, or maybe make a small note on the global flags section on each of the options in docs that this option is f.ex. only useful in a copy or sync scenario.
I’ve also tried to use some options that simply fucks up the the whole job like setting the backlog to a higher value. For some reason rclone just skipped all the files an didn’t copy any of them if I did that. Something like that would be noticed of course under test, but it would save everyone a lot of time if rlcone could just quit and say “hey, these options cannot be combined or should only be used in a spesific scenario”.
We had a discussion about sync’ing versioned files from s3 to s3, but the documentation doesn’t say anything about the fact that rclone doesn’t handle that in a standard way but rather saves the versions with postfixes. Even the AI’s didn’t agree on wether rclone handled versioned files or not.
I know rclone has a ton of options, and I’m not advocating for a full doc rewrite, but some options could be very useful to get some info or feedback on what main command or scenario they apply at all to.