Using rclone as library for mobile application (authz issue)

Hi,
I'm working on application which will run on desktop and mobile (based on fyne io). For this application I need to sync files from different cloud providers and I was able to plug rclone into my source code for that purpose. Everything is working fine on desktop side, but I face one issue on mobile platform which is related to OAuth calls made by rclone. The rclone performs an HTTP call to certain provider, e.g. Dropbox, by using skratchdot/open-golang library. This library allows to open given HTTP request in default browser to perform authorization process. But this library does not provide open functionality on mobile platform (I tested it on Android). As such, I am unable to authorize my app when I call rclone configfile.Install() call on mobile platform. I already filed a github issue about it, see Please provide support for mobile browsers · Issue #32 · skratchdot/open-golang · GitHub

But, I also tried to use another approach by making my own authorization function (via fyne io OpenURL call. This allowed me to obtain token on mobile and write it to rclone.conf But this time I faced another issue where rlcone fails to access my Dropbox path using my app's token with the following message:

RCLONE_CONFIG=$PWD/rclone.conf rclone sync dropbox:ECM local:/tmp/ECM

2022/06/10 12:41:14 ERROR : Dropbox root 'ECM': error reading source root directory: directory not found
2022/06/10 12:41:14 ERROR : Local file system at /tmp/ECM: not deleting files as there were IO errors
2022/06/10 12:41:14 ERROR : Local file system at /tmp/ECM: not deleting directories as there were IO errors
2022/06/10 12:41:14 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2022/06/10 12:41:14 ERROR : Dropbox root 'ECM': error reading source root directory: directory not found
2022/06/10 12:41:14 ERROR : Local file system at /tmp/ECM: not deleting files as there were IO errors
2022/06/10 12:41:14 ERROR : Local file system at /tmp/ECM: not deleting directories as there were IO errors
2022/06/10 12:41:14 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2022/06/10 12:41:14 ERROR : Dropbox root 'ECM': error reading source root directory: directory not found
2022/06/10 12:41:14 ERROR : Local file system at /tmp/ECM: not deleting files as there were IO errors
2022/06/10 12:41:14 ERROR : Local file system at /tmp/ECM: not deleting directories as there were IO errors
2022/06/10 12:41:14 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2022/06/10 12:41:14 Failed to sync: directory not found

The errors seems to be related to token I obtained for my own app rather the rclone one because I can perform sync successfully with rclone token. Since I found in rclone code that it hard-codes clientid and client secret for dropbox provider I think there is somewhere misconfiguration which does not allow rclone to use token for my own app.

Therefore, I'm seeking solution for mobile application which should be resolved by either of two routes:

  • provide ability to open URL on mobile platform, but it requires changes to upstream library, or
  • provide rclone library custom token for another app to use. So far I didn't find anything wrong with rclone setup which works when it uses its token generated by rclone authorization call but it does not work when I used my app's token in its config. Please note that I configured my app to access single folder on dropbox but I gave app permission to read/write files and metadata. This may or may not be related to rclone failure.

I would appreciate any advise how to proceed.
Thanks,
Valentin.

You can provide your own - you need to use the same ones you did the oauth with. See the client_id and client_secret in the docs.

It would be nice to do this. However rclone returns the URL in the config interaction so you could open it for the user or let them click on the link.

Nick, thanks for advise. I tried to use my client_id and client_secret to setup new config entry and I'm still experience the same issue. I added to my config client_id, client_secret and successfully authorized with my app. I put the same redirect_uri as rclone does and was able to see success message in a browser, and get the token. Here is what I have in rclone config now (stripping tokens and id/secret values):

[local]
type = local

[dropbox]
type = dropbox
token = {"access_token":"xxx","token_type":"bearer","refresh_token":"yyy","expiry":"2022-06-11T14:31:39.333861-04:00"}
env_auth = true

[ecm]
type = dropbox
client_id = xxxyyyzzz
client_secret = xxxyyyzzz
token = {"access_token":"xxx","token_type":"bearer","refresh_token":"yyy","expiry":"2022-06-11T14:50:12.760096-04:00"}
env_auth = true

And if I use rclone sync dropbox:ECM local:/tmp/ECM everything works fine, i.e. data is transfered. But if I use rclone sync ecm:ECM local:/tmp/ECM then I get the following error:

2022/06/11 10:52:01 ERROR : Dropbox root 'ECM': error reading source root directory: directory not found
2022/06/11 10:52:01 ERROR : Local file system at /tmp/ECM: not deleting files as there were IO errors
2022/06/11 10:52:01 ERROR : Local file system at /tmp/ECM: not deleting directories as there were IO errors
2022/06/11 10:52:01 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2022/06/11 10:52:01 ERROR : Dropbox root 'ECM': error reading source root directory: directory not found
2022/06/11 10:52:01 ERROR : Local file system at /tmp/ECM: not deleting files as there were IO errors
2022/06/11 10:52:01 ERROR : Local file system at /tmp/ECM: not deleting directories as there were IO errors
2022/06/11 10:52:01 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2022/06/11 10:52:02 ERROR : Dropbox root 'ECM': error reading source root directory: directory not found
2022/06/11 10:52:02 ERROR : Local file system at /tmp/ECM: not deleting files as there were IO errors
2022/06/11 10:52:02 ERROR : Local file system at /tmp/ECM: not deleting directories as there were IO errors
2022/06/11 10:52:02 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2022/06/11 10:52:02 Failed to sync: directory not found

I have no idea why dropbox source works and ecm one does not. Please note though I setup my app to have Scoped App (App Folder) ECM, i.e. it should have only access to ECM area. So may be this affects how rclone sync works.

Any further advice?

If your Auth only lets you look in the app area then that's all rclone will be able to see too.

Try

rclone lsf ecm:

To see what files you can see

with rclone lsf dropbox: command I see my full Dropbox area, all folders and files, while with rclone lsf ecm: I see nothing. As I wrote my app in Dropbox was configured to see only single App Folder (the ECM). So the issue with current setup is seeing the root which is visible in dropbox setup which see entire dropbox directory structure, and I see nothing in ECM case. On Dropbox the ECM area exist. Any other suggestions? Is there any debug flag to log the operation in rclone, e.g. see how it defines the root path?

And, now I can confirm that if I create an app with full access to Dropbox it works fine, while if I only constrains it to specific folder it does not. So, I think it is a bug either in configuration or missing permission attributes.

It sounds like the Dropbox permissions are working as they should.