Cache folder IDs from Box


#1

Hi,

The box remote, both with and without the cache, seems to have to perform an API call for each directory level from the top. So, if I am uploading a file to /this/is/a/long/path/to/a/deep/directory/, we have to look for /this, get its ID, get /this/is/, get its ID, etc… and these lookups take longer than many of the actual transfers.

If we were to keep the folder IDs in cache, when we need /this/is/a/long/path/to/a/deep/directory/, we could identify a directory along the path (say /this/is/a/long/path/to/a/deep, use its ID to check if its parents are still “a”,“to”,“path”,“long”,…, and then proceed from there) and skip the intermediate lookups.

Is this generally useful enough to be worth doing?


#2

The box remote does keep a directory cache, but it rebuilds it every time - you’ve seen it doing that.

As far as I’m aware there isn’t a better way of doing this as box doesn’t have a turn this path into a directory ID API. I’d love to be proved wrong though!

In general rclone tries to avoid local state as a matter of principle…

Some ideas

  • rclone could take an directoryID which it takes as the root directory. The drive backend does this already.
  • depending on what you are doing you could run a persistent rclone and use the API

That second point might be worth exploring…

What you do is run rclone rcd --rc-no-auth as the server, then run rc commands from herehttps://rclone.org/rc/ - you can copy files like that and rclone will be continuously running so no need to re-auth or fetch directory IDs.