Auto detect download limit/api/issue with union remotes

Feature
#1

I suspect this has been requested before but my searches didn't find anything (probably just searching for incorrect terms.)

It would be amazing if rclone could detect api/download limit/misc issues on a remote, specifically with union remotes, and exclude that remote for a timeout period or some similar mechanism.

I've been trying to automate this by grepping logs and executing scripts based on certain text triggers. It works just "okay" but it's not as accurate as I'd like and unfortunately I've hit my "coding" skill level limit (translation: I googled and mashed up a bunch of things in a very, very ugly way.). :grin:

Thanks for your consideration and thanks for making and developing this awesome utility.
Ed

1 Like
#2

Yes this has been asked for before.

It is probably about time we made an new issue on github for this - do you fancy doing that?

Suggestions on heuristics for rclone to use to mark a remote as down would be useful too.

@random404 can you add your suggestions too since I know you are interested in this feature

2 Likes
#3

Never done it before but I'll give it a shot! Thanks!

#4

Hope I did it justice :slight_smile:

1 Like
#5

Thank you for making that :slight_smile:

#6

Thank you for considering it!

#7

I have been needing this for a long time, but I had to stop using the rclone union and move to other setups I can't disclose to workaround it.

My proposition would be for:

  1. make the random open policy be actually random
  2. If a file fail to open, try to find the file in the next remote and open it, keep doing it till you give up or the file opens
#8

And I just a idea for 3) combined with a random search policy, you don't even need to see if file exists in the remote if it fails to open, since you have already that. can recycle that to cut api requests

#9

@random404 do you want to add those ideas to the issue?

#10

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.