Mount events API

Feature
1

Not sure if this has been discussed as yet, and I’m aware you scrape logs for events like new files being found on remotes (like polling google drive) - but was wondering if there’s any appetite for proxying such events to an API that rclone could be polled on.

Right now I have some tooling like https://github.com/l3uddz/plex_autoscan#google-drive-monitoring which can poll remotes for updating plex, great - except using crypt as a wrapper as you’d expect prevents this working.

It’d be useful if tools could use rc to poll for events like new files, deleted files, etc and trigger actions accordingly.

I’m sure there’s non plex usecases for having rclone report these events though and would love to hear any ideas where this would be useful to trigger things.

2

Can you try to come up with what you’d want an rc call to do? That might be the best place to start.

3

Since we are discussing plex here, I was thinking something like a websocket connection which sends messages for the following events:

  1. Create
  2. Move
  3. Rename (Can probably be merged into 2)
  4. Delete
4

I guess it would be something along the lines of rc/events where you are returned a bunch of filesystem events like google drive reports when you poll that for cache eviction and updates.

Essentially something to poll for events as they happen upstream from the mount.

The gist being is that using crypt with any mount makes the native API for events less useful as it’s all obstructed, the functionality would bridge the gap and allow apps to check rclone for events rather than gdrive etc.

5

So a call called rc/eventswhich you would poll for events from the mount? Or from drive itself?

6

I guess realistically you’d want to proxy events from the drive under the mount and use decryption to proxy out useful information about the files.

Not sure if it’s better to have the mount events, which would include new files being added from that machine earlier than the cloud itself (cache upload timer) or just the drive, so even if the file is available it still waits till it’s uploaded first.

The latter is probably more universal and makes it not reliant on the cache layer.

Thoughts?

7

Would being able to decrypt the drive changes with rclone perhaps be more helpful?

You can so this with rclone cryptdecode - that could be added to @spicypixel 's script.

rclone cryptdecode returns unencrypted file names when provided with
a list of encrypted file names. List limit is 10 items.

If you supply the --reverse flag, it will return encrypted file names.

use it like this

	rclone cryptdecode encryptedremote: encryptedfilename1 encryptedfilename2

	rclone cryptdecode --reverse encryptedremote: filename1 filename2

Usage:
  rclone cryptdecode encryptedremote: encryptedfilename [flags]

Flags:
  -h, --help      help for cryptdecode
      --reverse   Reverse cryptdecode, encrypts filenames
8

Yeah that’s a legitimate option, I’ll ask the developer if they’re open to doing that.

Thanks anyway!

Edit: I would then say would it be feasible to have an rc/cryptdecode call? Instead of giving over rclone crypt phrases or having to provide rclone config location would it make sense to be able to send an encrypted path to the rc instance and get back the decrypted path? I figure this is the most secure way so that no keys are needed to decrypt merely authenticated access to the rc interface.

Since I run rclone on its own user with very locked down permissions on the rclone.conf I guess I’d have to allow reading from the scripts user to allow them to use rclone against the crypted remote correct?

Just clarifying.

9

Yes that would be possible.

If you weren’t using an rc interface then read access to the config file would be required.

10

How about an ability to post web hooks on remote changes (renames/deletes/additions/moves) ?

then start rclone mount with a flag (such as --wh http://webhook-receiver.com/)

11

Hmm… I could do a remote control interface you could poll for changes - would that be interesting?

12

Definitely, that would be an ideal option as well.

There would need to be some sort of consistent marker for the changes, like Google’s page token.

13

Do you want to please make a new issue on github about it? Maybe you could help implement it?