When using 'rclone rc' commands somehow return errors as parsable json

What is the problem you are having with rclone?

When an 'rclone rc' command is sent the server error response can't be parsed. Right now it writes something like this so stderr:

Invalid method:

2023/09/19 09:46:02 Failed to rc: failed to read rc response: 404 Not Found: {
        "error": "couldn't find method \"cause/error\"",
        "input": {},
        "path": "cause/error",
        "status": 404
}
09/19 09:46:02 ERROR : rc: "cause/error": error: couldn't find method "cause/error"

Missing section:

2023/09/19 09:57:38 Failed to rc: failed to read rc response: 500 Internal Server Error: {
        "error": "didn't find section in config file",
        "input": {
                "fs": "SOURCE:/docker/",
                "mountPoint": "/tmp/src"
        },
        "path": "mount/mount",
        "status": 500
}
on in config file

It would be good to be able to actually parse the JSON error, maybe a flag we could set. We know the command failed (exit code), the stderr could be let as is for logging or something, but the actual json response could be send to stdout so we can parse it and know what happened. That way we could do something like:

OUTPUT=$(rclone rc command/with_error 2> /log_or_dev_null)
EXIT_CODE=$?
if [ $EXIT_CODE -ne 0 ]; then
    error_message=$(**get error field from json output**)
    ** pretty show error **
else
    ** do something with valid response **
fi

Maybe even make non-server errors also create a parsable response as well, like if the server is not running:

send to stderr (what is sending now):

2023/09/19 10:17:55 Failed to rc: connection failed: Post "http://localhost:5572/cause/error": dial tcp 127.0.0.1:5572: connect: connection refused

send to stdout:

{
        "error": "connection refused to http://localhost:5572",
        "input": {},
        "path": "cause/error",
        "status": 500
}

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 6.2.0-32-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.20.6
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Not important

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Not important, any command that causes an error like

rclone rc cause/error

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

Not important

A log from the command that you were trying to run with the -vv flag

Not important

That seems like a reasonable idea to me.

Want to have a go at it?

I would if I had any experience with go. I don't mind looking at it but I personally would want me messing with code I don't know with a language I don't know. :slight_smile:

Well it wasn't that hard, just did some changes to rclone/cmd/rc/rc.go and it seems to be working.
new flag '--error-output', will always output a json formatted error to stdout if the command fails (--no-output will also stop this). Will output even if server is not running, status code will be 400 (client error)

Anyway have no idea how to upload or send patches (never contributed anywhere) so how do I send it? Can I post the code here?

Well placed it in Google Drive maybe someone can download it and test it.

Link

Great work :slight_smile:

I tidied that up a bit and made a beta from that - please give it a go!

I decided that we always want the error outputted as a JSON blob so I removed the flag.

v1.65.0-beta.7378.932ebbf31.fix-rc-error on branch fix-rc-error (uploaded in 15-30 mins)

You can see the commit here rc: always report an error as JSON · rclone/rclone@932ebbf · GitHub

Note I've put a Co-Authored-By header in with your name which is traditional. Let me know if that is OK.

Thank you

I tidied that up a bit and made a beta from that - please give it a go!

I decided that we always want the error outputted as a JSON blob so I removed the flag.

I added the flag to not change existing installs/scripts using rclone, people expecting no output now get it. I agree with you that we should get a response.

v1.65.0-beta.7378.932ebbf31.fix-rc-error on branch fix-rc-error (uploaded in 15-30 mins)

You can see the commit here rc: always report an error as JSON · rclone/rclone@932ebbf · GitHub

Nice, tested it and it works, before tried using the 'install.sh' script, but it downloads a different one. Anyway usually how long it takes until the patch goes in the actual 'beta' that install.sh uses?

Note I've put a Co-Authored-By header in with your name which is traditional. Let me know if that is OK.

Thank you

It's ok, no problem with me.

Thanks for testing.

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.65 :slight_smile:

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.