Rclone OAuth Implemented Incorrectly. Exposes Client Secrets?

Help and Support
1

What is the problem you are having with rclone?

I am investigating the possibility of enabling Rclone OneDrive for an organization. In doing so, I've been reading and trying to better understand the OAuth Implementation. I became confused by how Rclone appears to be handling the Remote configuration for obtaining a token from Microsoft Azure AD. I may be misunderstanding something, but it appears Rclone is handling its OAuth implementation incorrectly, lending users to expose their (Application) client secret(s).

Please correct me if I am misunderstanding something, because I am just learning about this I may be wrong. Also, I really want to include some links in this post to some references (Rclone documentation & Okta (OAuth) documentation, but I am apparently not allowed to post links :cry:

EDIT 1 I have now including links within context.
EDIT 2 I have found this older post referencing basically the same sort of issue (Is there anyway to handle this more permanently so refresh tokens can be obtained?): Best practice for keeping OAuth2 client_secret confidential

  1. Rclone is implementing their application as a "Web App" or in-terms of OAuth is considered a "Confidential Client". Confidential Clients require to register a client_id and a client_secret with an Authorization server.

Confidential clients are applications that are able to securely authenticate with the authorization server, for example being able to keep their registered client secret safe.

Public clients are unable to use registered client secrets, such as applications running in a browser or on a mobile device.

Why is Rclone implementing their application this way? It seems that rclone is actually a "Native App" or "Public Client" because it is storing the client_secret locally in a config file and is not able to keep this secret secure.

  1. Rclone has their Application (obscured) client secret listed within the source code. Since it is obscured it is easily reversible to reveal it. According to Okta (OAuth)

If the developer is creating a “public” app (a mobile or single-page app), then you should not issue a client_secret to the app at all. This is the only way to ensure the developer won’t accidentally include it in their application. If it doesn’t exist, it can’t be leaked!

Unfortunately for anyone that implements Using their Own Client ID & Key, the key is NOT obscured/encrypted and it is listed in plain text in the rclone.conf file

  1. This implementation causes a waterfall of other issues for organizations attempting to enable Rclone with their Tenants.
  • What is the point for Using your Own Client ID & Key with the current implementation?

    • As an organization, we would be creating our own client_id & client_secret that would have to be publicly available to each person that wants to connect Rclone to their OneDrive account. It seems pointless to have a secret that is publicly shared.

    • Rclone only implements the Microsoft Common authorization request(s). ...//login.microsoftonline.com/common/oauth2/v2.0/token which means the only option for "Account Types" is to allow signing into "our Tenant's" app with any tenant. It really would be ideal if Rclone could implement support to allow users to supply their own TenantName/TenantID to allow for better control over the app (e.g. ...//login.microsoftonline.com/{TenantID})

    • The only thing I can see it doing is helping to avoid potential throttling.

I was going to open this as an Issue but want to obtain feedback from the community first to make sure I'm not completely misinterpreting things.

What is your rclone version (output from rclone version)

Rclone v1.50.2

  • os/arch: linux/amd64
  • go version: go1.13.6

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20.04.1 LTS (Windows Subsystem for Linux)

Which cloud storage system are you using? (eg Google Drive)

OneDrive

2

hello and welcome to the forum.

rclone can encrypt the contents of its config files
https://rclone.org/docs/#configuration-encryption

that is a very old version of rclone, you should update to v1.53.1

3

Hello, and thank you for the response.

Unfortunately the version does not come into play much here because the source code and documentation I’m referencing are current.

Yes, users can manually encrypt the config files but this is still only optional. Also the Rclone default client secret is already exposed in the source code (just obfuscated).

Besides, my larger point that it appears there is no need for a client secret to be stored in the config file to being with. It seems that the OAuth implementation is just wrong.

4

there are a number of posts about the tentant issue.

maybe this can help

5

Thank you! That could be very helpful. I’ll definitely need to try that for the tenant issue and hopefully allows me more some control overall. It’s too bad that is not documented, if it works maybe I’ll suggest they update their docs.

Do you have any other thoughts on the OAuth implementation issue? To me it seems that it should be implemented as a public client instead of a confidential client.

6

sure, you can suggest an doc update but really, it is just a search away in the forum

not sure what the problem with storing the secrets in an encrypted rclone file?
but if there is a more secure way to get it done, sounds good to me.

as for the public versus confidential client, i would not have a clue.
but we have plenty of experts in that kind of thing and i am sure they will stop by soon.
in the mean time, perhaps consider https://rclone.org/donate/

7

Unfortunately, we have that in place so folks don't sign up and spam as it was an issue. You should be able to post links now.

8

Thank you, I appreciate it. I was able to update the post with links.

9

If you don't have a client_secret then you have to do the oauth login every time the token expires. Typically this is 1 hour. Rclone could work like this but it would be very inconvenient!

As far as I know you either need

  1. To store the client ID/ client secret in the application or
  2. Have an external service which stores them and this is called with the users credentials to sign them as part of the oauth process.

We've gone for option 1. Option 2. opens up a different problem in that the users credentials have to go through a service run by us. This wasn't a risk that we wanted to take, having everyone's credentials come through a service run by us, and it isn't something most users want either. It might be an option for you though, and I do have some code which we ran for a while for amazon cloud drive which does it: https://github.com/ncw/oauthproxy

Note that in the scale of things the oauth client_id and client_secret aren't particularly important. All they do is identify the app connecting. They do not allow access to data on their own. The user still needs to provide authentication to access the actual data. It is a layer in the security Onion, but there are better layers just beneath!

If the client_id and client_secret were leaked then someone would be able to pretend to be rclone. Which gets you access to no user data without user credentials.

Note that native apps using oauth2 pretty much all use the rclone solution. I've seen threads where people have installed an https proxy and read the oauth client and secret from lots of commercial apps.

We found a solution for this here

Some of the learnings for that need to go into the docs!

10

@ncw First of all, thank you for the response and your work on Rclone! I appreciate it.

The access tokens typically expire within 1 hour.
This is where Refresh tokens come in, which Rclone already handles.

I think you might be thinking about the Implicit flow for public clients (which is old and basically being deprecated). You are correct that "The implicit grant flow does not issue refresh tokens..."

Now the current spec and recommendation is to use both client types Confidential & Public (But especially Public) with Proof Key for Code Exchange (PKCE)

"Refresh tokens are, and always will be, completely opaque to your application. They are long-lived e.g., 90 days for public clients, but the app should not be written to expect that a refresh token will last for any period of time."

Here are the reference(s):

This reference/spec also shows where we simply need to perform the following:

  • Generate a code_verifier
  • Send a code_challenge and code_challenge_method in the initial Request for User Consent
  • Send code_verifier in the Request for Access Token

Here is another reference that really helps simplify the flow and steps even further and really helps clarify a lot of confusion (for me at least) regarding how Public clients can be handled given the circumstances.

In the end, it seems that you are correct in that whether Rclone is implemented as a Web Client or Public Client doesn't really matter because the App/Client can be impersonated in either case. (E.g. the client_secret is exposed, or doesn't exist, leaving no way for the Auth Server to truly confirm the client's true identity). This bothers me because there is so much information out there about separating Public Clients from Web Clients, it makes me think I could still be missing something. Either way, PKCE is mostly introduced to further harden all OAuth 2 apps to help prevent Tokens from being stolen and easily used by attackers. I still think switching Rclone to the Public Client and implementing things as recommended by the specification would help prevent confusion and even make configuration a bit easier on end users.

Please let me know what you think?

I guess I'm documenting the following more for myself as future reference:

I'm starting to understand why this is/was quite confusing as I continue to read more and become more familiar. Here are some examples of questions that represent the confusion:

11

The medium post was really useful - thanks! And thanks for your helpful notes.

This was also my understanding after reading the above. I've always been unhappy about embedding the client_secret in the native binary- trying to keep it secret is security theatre with no actual security benefits. This is explained quite nicely in the medium article. Not having a client_secret at all seems like exactly the right choice to me. The client_id is public and if the server is making security choices on that then that is a mistake.

This is exactly the same problem that DRM tries to solve and it is fundamentally impossible to make it secure without some part the user cannot inspect.

I note that some of the providers (eg box) implement the protection that you can't use a refresh token more than once. This means that for those providers you can't just copy the config file and have rclone work.

Yes PKCE looks like a worthwhile upgrade for rclone.

I found a bit of go code implementing PKCE

However the snippets in the medium article are probably easier to integrate into rclone's current oauth flow.

Are you interested in helping add PKCE support to rclone?

The relevant code is here if you want to take a look?

Not that rclone integrates with 16 oauth providers at the moment and it is most likely that some of these support PKCE so whatever we do should not just apply to onedrive.

12

应用机密公开并没有什么安全问题,问题在于你同时暴露了token和应用机密.目前比如oneindex等网页程序都是明文保存的, 应用机密可以是数字证书形似的如果你对此感兴趣可以尝试

13

Thanks again for the responses! All of this is very helpful in clearing up confusion and obtaining a better understanding of these concepts.

Unfortunately not, although I'm probably capable. I just dont have the time right now to learn the language and all the other concepts involved.

Again, I really appreciate all your work on this incredible app.

Translating your response (with google translate) just to help others that may come across this:

There is no security problem with the disclosure of application secrets. The problem is that you have exposed both the token and application secrets. Currently, web programs such as oneindex are stored in plain text, and the application secrets can be in the form of digital certificates. If you are interested, you can try.

1 Like
14

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.