Can Rclone be used as a Library or an Api?

I want to use Rclone in my project but I don't want to use the Rclone exe by command prompt because in that I have no control in my code like what input it is asking or there are different configurations for different remote storages. So is there any way I can do the whole tasks which I do by CMD from code itself like setting the whole could system? For eg. like when setting an AWS storage I need to provide many inputs to it like secret key, region, etc. I want to do this whole thing by my code.
API or Library will be fine but I don't know how to do that.

1 Like

Hi Aman,

Yes, I suggest you take a look at the rclone Remote Control:

It can be called by HTTP requests or directly from code as mentioned in this post:

Hi, thank you for replying. I checked this post. Can you please share some sample calls with me like creating a new config?

First please tell me some more about your situation:

Do you know how to use a REST API? used one before?
Did you try the examples in the docs? or searching the forum?
What is your background in programming?
Which programming language and tools are you using?
What are you trying to make?

Perhaps you don't need a library or API, perhaps you are just looking for the rclone config create command?

Example:

rclone config create mySFTP sftp  host=mySFTPserver.com port=22 user=myUser pass=mySecret

You may also want to use the --non-interactive flag or a combination. You can read more here: rclone config create

Here is how to create the above remote with the rclone API using VS Code with the REST client extension:

First start the rclone Remote Control Demon in a terminal:

rclone rcd --rc-no-auth

Then you can create and send the config/create request from an .http file:

POST http://localhost:5572/config/create HTTP/1.1
content-type: application/json

{
    "name": "mySFTP",
    "type": "sftp",
    "parameters":  { 
        "host": "mySFTPserver.com",
        "port": 22,
        "user": "myUser",
        "pass": "mySecret"
    }
}

You can read about the config/create request here: https://rclone.org/rc/#config-create

It looks like this on my screen:

Is this what you were looking for?

Hi, thank you so much.

  1. Yes I know and have used Rest API.
  2. I tried but am getting an authentication error while making calls from somewhere else even though no authentication is enabled.
  3. I am a working software developer.
  4. I am using the c# dotnet framework.
  5. I am trying to replicate the functionality of the Rclone Web GUI.

Yes, I was looking for something exactly like this. Thank you so much.

Perfect, thanks.

Sounds interesting, you seem to have all the skills needed. I see you also love Python, so you will probably find it easy to read the rclone source code in Go - if needed.

Tip: You can see the REST communication used by the web GUI if you start it like this:

rclone rcd --rc-web-gui -vv

Did you solve this?

Yes, I solved that error but now I am stuck at something else. When I making a post call like that you have mentioned I am getting an error

2022/10/03 12:46:24 ERROR : rc: "config/create": error: Didn't find key "name" in input

My call:

http://localhost:5572/config/create

{
  "parameters": {
    "token": ""
  },
  "name": "new",
  "type": "drive"
}

I am trying to add a Google Drive. Also, one more thing when adding Gdrive it redirects the request to the Google page, and when I authenticate it automatically detects that I have completed the request or it has got the token. How does it know that by web calls I also traced the network but could not figure it out?

Debug logs

2022/10/03 13:06:49 DEBUG : rc: "core/stats": with parameters map[]
2022/10/03 13:06:49 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletedDirs:0 deletes:0 elapsedTime:5.8939117 errors:0 eta:<nil> fatalError:false renames:0 retryError:false speed:0 totalBytes:0 totalChecks:0 totalTransfers:0 transferTime:0 transfers:0]: <nil>
2022/10/03 13:06:49 DEBUG : rc: "core/stats": with parameters map[]
2022/10/03 13:06:49 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletedDirs:0 deletes:0 elapsedTime:5.8939117 errors:0 eta:<nil> fatalError:false renames:0 retryError:false speed:0 totalBytes:0 totalChecks:0 totalTransfers:0 transferTime:0 transfers:0]: <nil>
2022/10/03 13:06:51 DEBUG : rc: "config/create": with parameters map[]
2022/10/03 13:06:51 ERROR : rc: "config/create": error: Didn't find key "name" in input

The parameters map is empty, so it seems like your json body wasn't passed to rclone (by your c# code, i guess)

This is my debug log when performing the exact same request directly from VS Code:

2022/10/03 09:36:16 DEBUG : rc: "config/create": with parameters map[name:new parameters:map[token:] type:drive]

This is the redacted browser call from my debug log:

2022/10/03 09:36:16 DEBUG : Starting auth server on 127.0.0.1:53682
2022/10/03 09:36:16 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=nK9dmMhHxtB9Nm_24Zv1Ew
2022/10/03 09:36:16 NOTICE: Log in and authorize rclone for access
2022/10/03 09:36:16 NOTICE: Waiting for code...
2022/10/03 09:36:16 DEBUG : Redirecting browser to: https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=202264815644.apps.googleusercontent.com&redirect_uri=http%3A%2F%2F127.0.0.1%3A53682%2F&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive&state=somethingsecrethere

Note, that &redirect_uri=http%3A%2F%2F127.0.0.1%3A53682... tells google to redirect the response to the auth server at 127.0.0.1:53682 stated at the top of the snippet.

I made the call using Postman and YARC. What code did you use for making the request?

I am using VS Code with the REST client extension:

Here is somebody else using Postman: Filter in RestAPI mode - #6 by Alessandro_Zatti, does you screen look similar?
(I never tried Postman myself)

Looks identical to me.

It's done, man. Thank you so much for your help. I traced my calls with the Wireshark and I found that I am not setting the protocol application/JSON.

Great, thank you very much!

Good work!

Perhaps you can share a quick screenshot showing how to set the protocol correctly in postman?
That would be a great help to future forum users.

I did this in postman:

Earlier in the highlighted it was text.

For c# code:

public static string MakeHttpRequest(Uri uri, string methodName, string json = "", string userName = "", string password = "")
        {
            string result = string.Empty;
            try
            {
                HttpWebRequest request = null;
                HttpWebResponse answer = null;
                Stream stream = null;
                StreamReader streamReader = null;
                NetworkCredential networkCredential = null;
                if (!string.IsNullOrEmpty(userName) && !string.IsNullOrEmpty(password))
                {
                    networkCredential = new NetworkCredential(userName, password);
                }

                request = (HttpWebRequest)WebRequest.Create(uri);
                request.ContentType = "application/json"; //this was the line that I was missing
                request.Method = methodName;
                request.Timeout = 50000;
                if (networkCredential != null)
                {
                    request.Credentials = networkCredential;
                }

                if (!String.IsNullOrEmpty(json))
                {
                    request.Method = "POST";
                    using (var streamWriter = new StreamWriter(request.GetRequestStream()))
                    {
                        streamWriter.Write(json);
                    }
                }
                
                answer = (HttpWebResponse)request.GetResponse();
                stream = answer.GetResponseStream();
                streamReader = new StreamReader(stream);
                result = streamReader.ReadToEnd();
            }
            catch (Exception ex)
            {

            }
            return result;
        }
1 Like

Thanks, I am sure none of us will ever forget that again :sweat_smile:

Happy coding!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.