Local writethrough with remote sync?

Feature
1

I’m wondering if a specific workflow is possible with rclone. I have a remote system that I’m using for building and running some code I’m working on. I’ve been using vim over SSH, but the latency is pretty killer. What I’d love to be able to do is edit files locally and have them synced instantly to the remote machine.

What I would love to be able to do is mount a FUSE folder locally that mirrors a remote folder. Any changes I make locally are immediately written though to the local file system, and then asynchronously synced to the remote. And remote changes are also automatically synced locally.

Is anything like this possible with rclone?

I’ve already looked into syncthing and lsyncd, but they aren’t designed for instantaneous synching.

2

welcome to the forum,

that also applies to rclone.

afiak, rclone mount is not desgined to be use that way.

using rclone mount, if you save a file, by default, it will be queued for upload in five seconds.

that will not work well with ssh.
need to use a remote that support remote polling, such as gdrive, onedrive and some others.

1 Like
3

Have a look at:

I am using it for exactly workflow you described. Remote system is PITA to work over ssh or some GUI remote access. I prefer my local editor and tools.

Works like a charm.

4

Thanks, that pretty much answers it.

5

I have used syncthing in the past. My main complaint is that files don’t sync immediately. I need a tight development loop where I save a file, switch to my SSH window, and run a compilation command on the remote machine, with the updated files already there.

6

i think this is as close to instant as you can get with rclone

  1. run rclone as a daemon using rclone rcd
  2. write a script in vim, every time you save the file, to call https://rclone.org/rc/#sync-copy using flags such as --no-check-dest --no-traverse
7

Not sure what is immediately for you but by default Syncthing syncs files after 10s (configurable). Either it was different with some old versions or specific to your OS where FS watchers did not work.

I have tried rclone as well and no configuration could beat Syncthing. rclone does not have filesystem notifications implemented which means that polling does not work for local FS.

8

Interesting that this is a major drawback for you @kapitainsky . Syncthing is written in Go too - I wonder how they do the filesystem notifications... GitHub - rjeczalik/notify: File system event notification library on steroids. is the answer (or rather their own fork) as mentioned in Feature request: Listen filesystem events and sync · Issue #249 · rclone/rclone · GitHub - maybe time to revitalise that issue.

1 Like
9

Only for very specific situation when I work on multiple source files locally but then have to test them on remote machine with problematic access. I have tried multiple ways and found Syncthing least problematic. Filesystem notifications triggered sync works rock solid but this is probably because of good OS support for such feature (macOS). I use filesystem notifications myself to trigger launchd jobs (systemd like thing) when specific file or directory changes. So for example I can run some out of the schedule backup when very important data changes.

On top of this I do not trust `rclone mount/nfsmount` 100% (at least on macOS):slight_smile: Especially when I have to edit files.