Rclone API rcd header for BOX API

Help and Support
1

What is the problem you are having with rclone?

I am using rclone api rcd with box enterprise but i am unable to find how can i pass the header
as:user as in rclone CLI we have the header flag to pass on to box but in rclone API i am unable to find that. I have tried to pass on the headers in _config but it is not working.

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1

  • os/version: Microsoft Windows 10 Pro 22H2 (64 bit)
  • os/kernel: 10.0.19045.3324 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.20.6
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Box

The command you were trying to run (eg rclone copy /tmp remote:tmp)

HTTP POST

http://127.0.0.1:5572/operations/list
Request Body
{
"fs": "box-remote:",
"remote":"",
"_config":{"headers": [{"Key": "as-user","Value": "00000000000"}]}
}

2

Could you please give an example how you do this in rclone CLI? It might help to figure out exactly what you want to achieve.

3

.\rclone.exe lsjson --header as-user:00000000000 box:/

4

and this one work as you expect?

5

Yes, how can be same achieved by Rclone API rcd

6

rclone rc operations/list remote="" fs=box-remote: _config='{"Headers": [{"Key": "as-user", "Value": "00000000000"}]}'

7

Running above command with actual Box user ID gives the same error

2023/08/31 15:41:17 Failed to rc: failed to read rc response: 400 Bad Request: {
        "error": "key \"_config\": invalid character 'H' looking for beginning of object key string",
        "input": {
                "_config": "{Headers: [{Key: as-user, Value: 00000000000}]}",
                "fs": "box-remote:",
                "remote": ""
        },
        "path": "operations/list",
        "status": 400
}
8

Your error indicates some problem with command format. Your shell misinterprets your syntax.

I am not Windows user though but maybe related to how Windows shell understands ' and "?

When I run it on non Windows machine:

$ rclone rc operations/list remote="" fs=box: _config='{"Headers": [{"Key": "as-user", "Value": "00000000000"}]}'
2023/08/31 12:09:12 Failed to rc: failed to read rc response: 500 Internal Server Error: {
	"error": "error in ListJSON: couldn't list files: Error \"not_found\" (404): On-Behalf-Of User not found ([123 34 105 110 118 97 108 105 100 95 117 115 101 114 95 105 100 34 58 123 34 105 100 34 58 34 48 48 48 48 48 48 48 48 48 48 48 34 125 125])",
	"input": {
		"_config": "{\"Headers\": [{\"Key\": \"as-user\", \"Value\": \"00000000000\"}]}",
		"fs": "box:",
		"remote": ""
	},
	"path": "operations/list",
	"status": 500
}

you can see that it fails with response from box remote (it does not have 00000000000 user).

BTW:
123 34 105 110 118 97 108 105 100 95 117 115 101 114 95 105 100 34 58 123 34 105 100 34 58 34 48 48 48 48 48 48 48 48 48 48 48 34 125 125

converts to:

{"invalid_user_id":{"id":"00000000000"}}

Try from CMD, PS or script would be my suggestion on top of learn how to quote or escape some characters in your shell.

9

@asdffdsa - any insight form Windows specialist?

10 
 rclone rc operations/list remote="" fs=box-remote: _config='{\"Headers\": [{\"Key\": \"as-user\", \"Value\": \"0
0000000000\"}]}'

This is how the command should look like.
But I am getting another error now.

2023/08/31 16:26:13 Failed to rc: failed to read rc response: 500 Internal Server Error: {
        "error": "didn't find section in config file",
        "input": {
                "_config": "{\"Headers\": [{\"Key\": \"as-user\", \"Value\": \"00000000000\"}]}",
                "fs": "box-remote:",
                "remote": ""
        },
        "path": "operations/list",
        "status": 500
}

This is my Config Look like

[box-api]
type = box
box_config_file = D:\WorkXXXXX-box.json
box_sub_type = enterprise
token = XXXXXX

And here is the response from Postman call

	"error": "error in ListJSON: couldn't list files: Error \"not_found\" (404): On-Behalf-Of User not found ([123 34 105 110 118 97 108 105 100 95 117 115 101 114 95 105 100 34 58 123 34 105 100 34 58 34 48 48 48 48 48 48 48 48 48 48 48 34 125 125])",
	"input": {
		"_config": {
			"Headers": [
				{
					"Key": "as-user",
					"Value": "XXXXXXXXXXX"
				}
			]
		},
		"fs": "box-XXXX:",
		"remote": ""
	},
	"path": "operations/list",
	"status": 500
}

Two different responses.
I am new to RClone.
Is there a way to add user in box config as we can add in google drive config with key "impersonate"

12

That looks like it is working, you just need to put a valid user ID in. Those numbers translate to the same error message.

Thats probably a good idea! Can you open a new issue on Github and link to the box docs on how to use the header please?

13

I am going to reset everything in case I missed something.

Thanks for considering it. I will do it once I am done with current life crisis :grinning:.

14

you do not have box-remote.. you have box-api remote

15

Sorry for inconsistent remote names but I change them knowingly as my real remote name contains company name ids.
Also I am using a service account and not client_id and client_secret. Does that make any difference.

16

Even I have no clue what Postman is you get it right at the end.

You just do not have 00000000 user.

Error message is in ASCII - 123 34 105 110 118 97 108 105 100 95 117 115 101 114 95 105 100 34 58 123 34 105 100 34 58 34 48 48 48 48 48 48 48 48 48 48 48 34 125 125

you can convert it to text - {"invalid_user_id":{"id":"00000000000"}}

so now your command has correct syntax and error message from Box tells you what is wrong.

17

I pushed a little fix to the latest beta which turns that into

Error "not_found" (404): On-Behalf-Of User not found ({"invalid_user_id":{"id":"00000000000"}})

Which is a little easier to understand! I don't think I've seen one of those errors before so I guess I never noticed we were outputting the message as numbers instead of a string!

18

I have some findings which I would like to share.

You were correct I made a mistake there. Sorry about that.

That will be helpful indeed.

Now, there are some issues with current RClone Box backend which I observe with api server. Google backend works fine BTW.
It is somehow cashing the input and results. I only passed "00000000000" may be once or twice because I was having JSON format issues but server cached it and was returning the same result even when I was passing a correct Box user id. I had to restart the server to get the correct results.
In my case caching is not a friend because I need to perform multiple migrations at the same time.
A solution is to add AS-USER key in box config and not using cache.
As a newbie I am unaware of the project philosophy so my solution may not be for all :pleading_face:.
We are currently making changes internally in box.go to work in our case.
It will be a long journey as I am starting to work with Go from this project. I am from Python/Django background.

PS: I am going to create the issue on github

19

That will depend on exactly what you are doing. If you are creating a new backend with the extra config then yes backends are cached for 5 minutes by default.

This won't be a problem if we make the impersonate flag as this will become part of the cache key and a different backend will be created.

Looking forward to seeing your issue :slight_smile:

20

thank you @Abdullah1 for raising the issue and @ncw for implementing it.

"fix-7267-box-impersonate" works great with the latest rclone beta and hopefully helps with box' API limits per user.

any idea how to implement a round-robin feature to add more than one ID to impersonate as-user?

either shuffle per default or after receiving a certain error count? - not sure if this would be possible based on box' API implementation.

21

As per my understanding, you want to add multiple box_user_ids to impersonate as-user in a single config section.

Box APIs only support a single as-user id in headers so it is not supported.
I suggest that you use List Enterprise User to get all users and create config for all of them using same credentials file and respective user_id.