Hi everyone,
I'd like to propose a new command for rclone: rclone changes. This command would leverage a "ChangeNotify" mechanism (common in many backend implementations) to efficiently retrieve a list of paths that have been modified since a given point in time.
Background:
While experimenting with building my own rclone backend, I encountered a useful feature called "ChangeNotify." This mechanism allows querying for changes (creations, modifications, deletions) that have occurred within a storage system after a specific timestamp. I understand that rclone uses something similar internally; for example, the mount command uses it to keep its view of the remote filesystem up-to-date efficiently. ChangeNotify can be a valuable tool for efficiently tracking changes in a storage system.
Proposed Command:
I'm suggesting exposing this functionality as a command-line tool: rclone changes.
Example Usage:
rclone changes GoogleDrive: --from "2025-03-29T08:49:56.205Z" --json
Expected Output:
The command would output a JSON list of file paths that have changed since the --from timestamp. For example:
[
{ "path": "path/to/file1.txt" },
{ "path": "path/to/directory/file2.jpg" },
{ "path": "another/file3.pdf" }
]
Use Case & Benefits:
My primary use case involves syncing metadata from a remote to a destination remote. I only need to sync the metadata, not the file content itself. Currently, I use rclone lsjson to get a listing of the source remote and then manually compare it to the destination. I can't use the rclone sync command directly because I don't want to sync the file contents.
To keep the two remotes in sync, I have to run lsjson periodically and compare the results. This is inefficient, especially when relatively few files have changed.
The rclone changes command would provide a much more efficient way to identify only the files that need to have their metadata updated on the destination. Instead of a full listing and comparison, I could simply:
- Use
rclone changesto get a list of modified paths. - Update only the metadata for those specific files on the destination remote.
This would significantly reduce the overhead of keeping the two systems synchronized, especially for large remotes with infrequent changes.
Am I Missing an Existing Solution?
I also wanted to ask: Is there already a way to accomplish this with existing rclone commands or configurations that I'm simply overlooking? I'm open to the possibility that I might be misunderstanding the best way to use rclone for my use case (syncing metadata only). If anyone has suggestions or alternative approaches, I'd greatly appreciate your insights!
I'm interested to hear your thoughts and feedback on this proposal! Thanks for considering it.