I would love to have a go (pun intended) at this but I do not have the skills to work on a go project. Back in 2006 I was the first person to use the then newly announced Amazon S3 to create an infinitely large disk device (that never breaks). (see https://forums.aws.amazon.com/thread.jspa?threadID=10271). So basically a precursor to "rclone mount". All that was written using a combination of C and Python. Separately I worked on an erasure code project. I never put the two together. I might be able to help with the design and maybe I can do some beta testing. Have you asked Klaus Post if he is interested to undertake this?
I would suggest not making this part of the UNIONfs remote since it serves different purposes. The UNIONfs is about merging different cloud storage into one bigger one. The erasure-code remote (M out of N where M<N, and M>1) would do the following:
-
keep all of your data readable even if some clouds are unavailable (the other day both OneDrive and GoogleDrive had major outages (fortunately not at the same time)).
-
keep your data safe from snooping by one cloud provider. Rumors are that OneDrive is easily hackable. With the data erasure encoded you will need to hack more than one cloud to be able to read the data (M>1).
-
increase download speed. First of all because the data is striped and secondly we only need the first M "replies" to the request for download. So we effectively use the fastest responding clouds.
Let's keep designing this and I would love to hear from others if this would be of interest to them.