rClone on large scale Linux desktops + AzureAD auth

jal23 · August 5, 2020, 11:38am

Hi all,
We would like to employ rClone on large scale on our Linux Desktop (1k+ clients) at the company as we use MS OneDrive Business as main backup tool (not my choice) these days and I am working on getting feature parity with the Win/Mac desktop.

While testing rClone with our IAM expert, we ran into a couple of AzureAD related issues. In a nutshell, we could not figure a decent way to enable (global) access to rClone. We followed the guide and the setup recommendations but we did not find a good way to enable globally rClone access - only on individual base (meaning we would need to put every user on a positive list) which is not really an option at the moment. We also tried building a new app ID + client secret and share this among several users but this does not work (or we did something wrong here) - nevertheless, this should not be the way this is intended to be used.
Has anyone made some similar experience? Is there a large scale deployment producition use? I am happy about any success story or hints!

ncw · August 5, 2020, 8:53pm

MS Auth is very complicated and I'm not an expert but maybe we can work something out together.

So what you'd like if I'm understanding you is for rclone to authenticate itself with AzureAD on behalf of a user and then get access to the users OneDrive space

I guess you don't want each user to have to do an oauth.

I don't understand quite what you mean here. Can you explain more?

I think regardless of the app Id you'll still need some user based Auth.

What do you think the best way of working for rclone should be?

jal23 · August 6, 2020, 2:08pm

Hi Nick,

Thanks for getting back to me. Basically, we're trying to use rClone for the whole organization to sync files to OneDrive for Business. Each client to his/hers OneDrive profile. Currently, we need to allow access on a per user base in the AzureAD/MS auth backend. This is what we end up as a user trying to login:

ncw · August 6, 2020, 3:22pm

Ah, so I guess you've tried whitelisting the app itself by making a new client_id and client_secret and whitelisting the client_id? I think other Azure AD users have managed that.

jal23 · August 6, 2020, 3:57pm

yes, this is what we tried - generating a new client_id and client_secret and put this on a positive list in AzureAD. And using these data across in the rclone config across all users. But we did not understand whether (1) this is safe to be used (not to have any access breaches) and (2) it did not really work in a quick check.
If this is a way that is to be followed, I will it once more. Is it enough to have only the same client_id positive-listed?

ncw · August 6, 2020, 10:04pm

I think allowing the client_id is the way to go.

It really only identifies the app to OneDrive, it doesn't identify the user or anything like that so the security impact is quite low.

As for why it didn't work, I don't know...

jal23 · August 7, 2020, 3:01pm

we did try it again with a new app_id which worked on my client. But only with my client, trying on another instance (in a vm) using the same client_id (but again with empty app_secret) did not work. This app_id has admin consent granted. Anything further we missed?

ncw · August 7, 2020, 8:57pm

I think you will need the app secret too otherwise the app won't be able to prove what it is.

jal23 · August 10, 2020, 10:29am

tried both with a preset client_id and client_secret. Did again not work for more than one client. Also tried multi-tenant vs. single-tenant, did not make a difference. Not sure what else to try anymore :(((

ncw · August 11, 2020, 9:42pm

Do you mean it didn't work for more than one user?

Hmm... I think this should work, but I don't have access to as admin account to try things alas.

As far as I'm aware you should be able to make your own client_id and allow that throughout your org - that is what the client_id is for.

When you say it doesn't work, what happens? What does the log say?

jal23 · August 12, 2020, 10:22am

Yes, sorry, you're right. It worked for only one user. I ran a test, it worked fine for me, then my colleague tested with setting up rclone using the same client_id and client_secret but it did not work for him (using also different browser to avoid some cookie/token caching issue I also read about in similar forum entries).

I checked the log and I do see the following
Failed to create file system for "onedrive:": failed to get root: Get https://graph.microsoft.com/v1.0/drives/xxxxxxxxxxxxxxxxxx/root: oauth2: cannot fetch token: 400 Bad Request

Application 'xxxxx'(rclone-test) is not configured as a multi-tenant application. Usage of the /common endpoint is not supported for such applications created after '10/15/2018'. Use a tenant-specific endpoint or configure the application to be multi-tenant.\r\nTrace

So, as mentioned, our IAM admin also said that multi-tenant is no option and the internet suggest the very same.

asdffdsa · August 12, 2020, 1:38pm

hello,
what versions of rclone are you running?

D1Ck3n · August 12, 2020, 3:58pm

Hi @asdffdsa ,

i work with @jal23 on this problem.

i tested it with version 1.52.2

ncw · August 12, 2020, 9:27pm

Maybe we could try the use the tenant specific endpoint if the multi tenant is not an option?

I'm not sure how you configure that, did you know?

jal23 · August 18, 2020, 12:30pm

Hi Nick,

Thank you for your continuous feedback. We opened a case with MS and had a few calls with them. Eventually, it seems that the issues lies with the endpoint URL: rclone employs login.microsoftonline.com/common/oath2/ whereas the it should have the tenant ID login.microsoftonline.com/TENANT_ID/oath2/

I tested this is with the latest rclone v1.52.3

Is there any way to change/specify the endpoint URL?

ncw · August 19, 2020, 1:28pm

That is interesting.

Currently the auth and token urls are

			AuthURL:  "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
			TokenURL: "https://login.microsoftonline.com/common/oauth2/v2.0/token",

You can set these in the config file with auth_url and token_url if you wanted to give it a try. I would guess you probably need to set both.

You can use -vv --dump headers to see what rclone is connecting to.

Where does that TENANT_ID come from - is it your tenant ID? Should rclone be asking for the TENANT_ID to construct those URLs do you think?

Do you have some links to docs on this?

I'm very interested to see if that works for you!

jal23 · August 20, 2020, 12:46pm

it should be this document:

ncw · August 21, 2020, 4:38pm

Thanks

Did it work?

jal23 · August 24, 2020, 7:11am

sorry, did not have the time so far. Our IAM expert is also on holiday, too. I need him to create me another client_id and client_secret.

I assume in the rclone.conf I can adjust the variables auth_urland token_url? Any hint for the syntax? I tried it as follows. At least with rclone config I was not able to run the setup with (I tried /usr/bin/rclone config -vv --dump headers)

And yes, your Azure Tenant ID can be looked up in the your Azure console. https://techcommunity.microsoft.com/t5/office-365/how-do-you-find-the-tenant-id/m-p/89018
Once this work, it would maybe make sense to ask for the tenant ID (at least for OneDrive Business).

ncw · August 25, 2020, 9:07pm

The syntax looks good for the config file. I'm not 100% sure you need the token URL so you could try commenting that out.

I think you may need the client ID and secret thus way unless you whitelist rclone's inbuilt client ID.