How to decode error response from server

[Trevor_Bastian]

What is the problem you are having with rclone?

I'm working on some software that synchronizes settings files with an AWS account, and occasionally things go wrong. It's fine - it's not specifically rclone's fault. AWS is usually nice enough to return an xml describing what went wrong - and I want to capture that.

I can use the flag --dump responses and that will show me exactly what I want to see in an error state - BUT if there is no error it ends up printing out a lot of binary data. Is there a way of only dumping that response in the case of a failing call?

the XML looks like this:

<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<Expires>2019-06-13T12:12:43Z</Expires>
<ServerTime>2022-08-11T22:20:22Z</ServerTime>
<RequestId>foo</RequestId>
<HostId>
bar
</HostId>
</Error>

Run the command 'rclone version' and share the full output of the command.

$ rclone --version
rclone v1.58.0

• os/version: ubuntu 18.04 (64 bit)
• os/kernel: 5.4.0-122-generic (x86_64)
• os/type: linux
• os/arch: amd64
• go/version: go1.17.8
• go/linking: static
• go/tags: none

Which cloud storage system are you using? (eg Google Drive)

AWS

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copyurl --dump responses '<url>' <dest>

The rclone config contents with secrets removed.

default

A log from the command with the -vv flag

$ rclone copyurl '<url>' ./junk.tmp -vv
2022/08/11 22:18:12 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copyurl" "<url>" "./junk.tmp" "-vv"]
2022/08/11 22:18:12 DEBUG : Creating backend with remote "./"
2022/08/11 22:18:12 NOTICE: Config file "/home/trevor/.config/rclone/rclone.conf" not found - using defaults
2022/08/11 22:18:12 DEBUG : fs cache: renaming cache item "./" to be canonical "/home/trevor/src/f-ecoation-ros"
2022/08/11 22:18:13 ERROR : Attempt 1/3 failed with 1 errors and: CopyURL failed: 403 Forbidden
2022/08/11 22:18:13 ERROR : Attempt 2/3 failed with 1 errors and: CopyURL failed: 403 Forbidden
2022/08/11 22:18:13 ERROR : Attempt 3/3 failed with 1 errors and: CopyURL failed: 403 Forbidden
2022/08/11 22:18:13 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.4s

2022/08/11 22:18:13 DEBUG : 2 go routines active
2022/08/11 22:18:13 Failed to copyurl: CopyURL failed: 403 Forbidden

[asdffdsa (jojothehumanmonkey)]

hello and welcome to the forum,

perhaps scan/parse the rclone log file.
in my case, for each run of rclone, a script will parse/scan the log file and email a summary report

[ncw (Nick Craig-Wood)]

@Trevor_Bastian

I can use the flag --dump responses and that will show me exactly what I want to see in an error state - BUT if there is no error it ends up printing out a lot of binary data. Is there a way of only dumping that response in the case of a failing call?

Not currently

Note that --dump responses should only be used for debugging as it stores the request in memory before logging it which means, for example, if you download a 1GB file it will be buffered in memory first when using --dump responses!

Do you think rclone should be showing more of the AWS errors? Rclone uses the AWS SDK and they will show the default way the SDK shows them which often isn't terribly helpful. There may be a way of getting them to be more verbose, I'm not sure.

[Trevor_Bastian]

Thank you all - I ended up just catching errors and making up my own response. Not a perfect solution - but it'll do for now.