Hey,
would rclone be interested in adding pricing information for each backend on the backend support page?
I would be willing to gather the initial data, maybe even write up a small javascript calculator so you can instantly see which backend is cheapest for your needs.
Kind regards,
ambiso
That is certainly an interesting idea!
It might be annoying to keep up to date.
The overview page might not be the best page for it - maybe it could be on its own page.
The input data could come from a google 'sheet and the release process could build the page from the 'sheet
Maybe rclone should have some affiliate links here too (clearly marked as supporting the project).
Some of the pricing structures for the providers are complicated though.
It might be annoying to keep up to date.
Certainly! But I feel like rclone could be the place where there is the most incentive to keep this up to date.
I guess the easiest prototype could be just a Google sheet? Although I think the data should be git tracked eventually, so you can issue updates via pull requests.
Affiliate links sound like a great way to support rclone devs.
Yeah, some of them are complicated. There should be some disclaimer that the calculator may be inaccurate, out of date or disintegrate upon use 
I really like this idea... I've been thinking about something similar for ages but haven't got off the notepad!
Starting with a Google 'sheet sounds like a good plan.
I'll just brainstorm a few ideas for things we might need in the sheet.
Basics
Some providers charge in chunks like you can get 1TB, 5TB, 20TB plans. How should we compare these to the providers that charge per byte?
What about currencies? Some providers only charge in $ or EUR, some will charge in any currency. Maybe keep the native currency and convert using a current exchange rate?
How do you see the UI working? The calculators I've seen you put in how many TB you want to store and how many GB you want to upload and download each month. That would work out OK. For the tiered providers we could pick a tier which would accommodate that amount of TB.
Some dropdowns for expected use to help set the parameters might be interesting, eg daily backup, video streaming, long term archive, other... So daily backup of 5TB of data - we can then estimate upload / download / transactions which nobody has a clue what to put in!
Things to watch out for - are the providers quoting per GB = 1,000,000,000 bytes or GiB = 1,073,741,824 bytes? For example s3 use GiB whereas b2 uses GB (reasonably sure about this).
Some providers offer different tiers hold, cold, infrequent, etc which it would be nice to get in.
Some providers offer bulk discounts.
I think whatever we do it will never be totally accurate, however we can get feedback of real world usage from rclone users which could help estimate the actual bills which will make it more accurate than most calculators when users just put in random data!
Maybe you can create a Google sheet so we can experiment with it a bit?
Basics [...]
I've never really used S3 and don't know about cost per transaction, maybe you can share a link for that pricing information?
There should be some distinction for business vs individual plans. Although individuals can sometimes just sign up as business customers, it doesn't work the other way around.
I think we should also add an option for how many months the storage is needed.
Many providers offer a discount for when you pay annualy. pCloud even has a lifetime offer.
Some providers charge in chunks like you can get 1TB, 5TB, 20TB plans. How should we compare these to the providers that charge per byte?
The calculator should probably choose the cheapest plan that can accommodate the data required.
What about currencies? Some providers only charge in $ or EUR, some will charge in any currency. Maybe keep the native currency and convert using a current exchange rate?
Yeah, for now I'd try to keep it simple and use a fixed exchange rate.
Some providers have their own conversions (i.e. Dropbox doesn't care whether you're paying in € or $, the price is the same for individuals).
How do you see the UI working? The calculators I've seen you put in how many TB you want to store and how many GB you want to upload and download each month. That would work out OK. For the tiered providers we could pick a tier which would accommodate that amount of TB.
The Backblaze B2 calculator (backblaze DOT com/b2/cloud-storage-pricing.html) has an option to select how much data you delete each month, and its accumulating the upload over time.
I'm not sure if that's something we want?
If we do want that, I'm not sure how to handle the plan selection for tiered providers: if we have 1000GB stored, and upload and delete 100GB/mo - should the plan allow for 1000GB or for 1100GB?
Additionally, if we allow for upload/deletions accumulating over time, then we have to consider, that a user might want to switch to allow for more or less data, to save costs.
So far I see a user providing the following information:
Maybe the upload/deleted/download could be used as an indicator for how much storage is hot vs. cold?
Some dropdowns for expected use to help set the parameters might be interesting, eg daily backup, video streaming, long term archive, other... So daily backup of 5TB of data - we can then estimate upload / download / transactions which nobody has a clue what to put in!
That sounds good, but I'd probably keep it simple for now.
Things to watch out for - are the providers quoting per GB = 1,000,000,000 bytes or GiB = 1,073,741,824 bytes? For example s3 use GiB whereas b2 uses GB (reasonably sure about this).
Yeah that's a problem.
I think whatever we do it will never be totally accurate, however we can get feedback of real world usage from rclone users which could help estimate the actual bills which will make it more accurate than most calculators when users just put in random data!
Definitely!
Ok I'll have a go with that and share it with you - PM me the email you want me to share it with.
I think all the prices should have a source URL - I'll put that in the sheet.
I'll try to put some providers in as an experiment
Yes that is a good idea.
It sounds quite complicated for users but it is a start.
Let's say it is just what you start out with at the start of the month rather than any max value.
Yes good idea. Though I think a price for both would be good.
Hot/Cold estimation based on upload/delete/download leaves a lot of room:
If you delete cold storage, and upload new stuff, the uploaded stuff will be hot in the next month.
However, if you upload/delete the same storage in a month, that upload won't be hot storage in the next month.
I'm not sure what the best solution is here - maybe you should be able to state what fraction of your upload/delete/download will be cold storage?
Also I'm reaching the limits of what is easily expressible in a spreadsheet quite quickly.
I think it might be better to switch to some code sooner rather than later.
this is a great idea and should stand on it own.
however, that will fail hard, when rclone, a tool that copy files, claims to have a super-duper 100% accurate calculator, that is forever forward always up to date and can compare all rclone supported cloud services in the millions of permutations.
it will only anger the same users who might want to support rclone by clicking on affiliate links.
Urg, getting complicated!
I guess that is the estimate of churn which the deleted files does.
Maybe, I think better in code anyway!
I think we are going to end up with some types of cloud provider in broad classes, eg s3, b2, gcs, azure blob, wasabi, hubic are all priced in roughly the same way, per GB/month, per GB/transfer, per transaction and have tiers. These are the "industrial" services where you definitely pay for everything you use.
Then there are the more "home" services which tend to be buy x TB for $y/year eg Google Drive, Onedrive, Dropbox, Box, pcloud etc.
Maybe trying to keep all this in a spreadsheet is a losing proposition!
When you say code what would be your preferred language? I'd probably reach for python as it is easy to write but it doesn't run on the web particularly well. Maybe it would make a good prototyping language.
Go would be a possibility since the rest of the project is written in Go. Maybe Javascript if we can keep it very clear would be the best solution. Javascript isn't a language and ecosystem I know particularly well though!
Good!
I'm not sure I'm following you here - is it the 100% accurate bit (it is never going to be and we've got to make that clear!) that is the problem or the ability to compare lots of providers at once or something else that will upset users?
Although python would be the language I know best, I think it would make sense to write it in javascript - with the other languages it would immediately become a hassle to run them on the web, or you would have to make requests to a server just to calculate some very simple formulae. I'm imagining this project can be a frontend only kind of deal - there shouldn't be a need to send any data anywhere.
I guess keeping it separate, developing it to some level, and waiting until the community can make a decision whether and how this should be included on rclone's page seems reasonable.
I think let's write the prototype in Python. We can convert it to something else later very easily. I'd rather have clear expression of ideas in the prototype phase rather than be fighting with the language and tooling.
Hi,
I've made a TypeScript prototype with Vue.js, and I've added a bunch of providers and very slowly it's coming together.
You can see the current absolutely abhorrent (design wise) prototype here:
ambiso.github DOT io/rclone_backend_pricing/
I'm particularly proud of how it uses a dynamic programming algorithm to determine the cheapest combination of plans to cover exactly the number of months with their respective storage requirements.
For example, if you need 10 months of storage, it will consider using the 6 months plan and 4 times the 1 month plan, or just the 12 months plan. And all these combinations are checked in O(number of plans * number of months) time!
However, so far, I've left out every provider with a non-obvious pricing model.
The pricing models, that I'm unsure how to approach are:
Also the compute cloud storage providers have much more complicated pricing models than others (for probably good reasons).
e.g. they have individual prices for every data center location.
I'm not sure if this should be reflected in the application - if you care about location, you probably have other (more important) requirements than price anyway? (vendor lock-in for example)
I could just choose the cheapest price of all data centers and use that?
I guess it's time to determine who this tool should be for.
For me it's just about wanting some bulk storage that I can mount with rclone.
I don't know what exactly others might be looking for.
What do you think? @ncw
Also, I've realized that 1fichier often comes up first in the results. However, I haven't had the smoothest experience with them - I think maybe we should also include a very short review of the providers, possibly also with regards to how well they interoperate with rclone?
OneDrive for Business, for example, has the bug, that it creates multiple versions of a file, whenever you create just one - this results in only being able to use half of your storage sometimes (you can't disable versioning with OneDrive for Business, and you can't easily delete the versions on many files).
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.