Local Database for Large Remotes

[asdffdsa (jojothehumanmonkey)]

imho, the beauty of rclone it is simple and stateless.

somewhere in the forum,
there is a script that uses rclone ls on the source and dest, creates a sql database table.
and from that, creates and --files-from

Efficiently update (copy) only very few files to Google drive

Background: see my previous question. While the challenge to accelerate rclone copy remains, I start a new thread as it is a distinct question. Test case: 10 k files on the server, 1 file modified (i.e. touched). The test case is approx 1/30 of my real use case. I looked into three options. See outputs below for details. direct rclone copy from local to encrypted gdrive - 12 minutes rclone mount --vfs-cache-mode full plus rclone copy from local to mountpoint - 45 minutes script to identify ou…