Cloud file provider APIs on Windows and MacOS

[vercas]

Both Windows and MacOS now offer APIs specifically for synchronizing files between local disks (NTFS and APFS only, respectively) and cloud storage.
This is Microsoft's API: Cloud Sync Engines - Win32 apps | Microsoft Docs
This is Apple's API: Apple Developer Documentation

Also, Apple announced the deprecation of kernel extensions: Deprecated Kernel Extensions and System Extension Alternatives - Support - Apple Developer
This puts the future of macFUSE in question. They specifically say that kernel APIs will be deprecated slowly as userland alternatives are provided. We have a userland API for file providers. Whatever macFUSE uses is clearly queued for the guillotine.

These APIs also provide nice-to-have features that FUSE doesn't have (on any platform): the ability to tell the user (via Windows Explorer and Finder) which files are hydrated, which aren't, as well as download/upload progress, and also the ability to pin files so they are always available offline.

I realize that implementing these will require quite a bit of effort. The Windows Cloud Files API is C/C++, which shouldn't require any rocket surgery to implement in Go (right....?). The MacOS API is only available in Swift and Objective-C. I expect this to be trickier but not impossible.

Sadly I don't know any Go (or Swift, or Objective-C) so I can't even begin to size the effort here.

[ncw (Nick Craig-Wood)]

@vercas

Both Windows and MacOS now offer APIs specifically for synchronizing files between local disks (NTFS and APFS only, respectively) and cloud storage.
This is Microsoft's API: Cloud Sync Engines - Win32 apps | Microsoft Docs

Good links thanks!

@vercas

This is Apple's API: Apple Developer Documentation

That looks like it could be implemented by an objective C app which uses rclone as a library.

@vercas

Also, Apple announced the deprecation of kernel extensions: Deprecated Kernel Extensions and System Extension Alternatives - Support - Apple Developer
This puts the future of macFUSE in question. They specifically say that kernel APIs will be deprecated slowly as userland alternatives are provided. We have a userland API for file providers. Whatever macFUSE uses is clearly queued for the guillotine.

Possibly, though I'm not sure that the sync api above obsoletes a full filesystem.

@vercas

I realize that implementing these will require quite a bit of effort. The Windows Cloud Files API is C/C++, which shouldn't require any rocket surgery to implement in Go (right....?). The MacOS API is only available in Swift and Objective-C. I expect this to be trickier but not impossible.

Linking Go with C is a bit of a pain. Win32 APIs are relatively easy to use from Go - but no idea about Objective-C.

I suspect the macOS solution would be to write an objective-c or swift app which uses rclone as a library. That might work for the Windows platform too. That is certainly how I'd prototype it anyway.

[vercas]

Developing file sync servers that use rclone as a library sounds like a sensible idea to me.
Does rclone even compile as a library, that exports C-compatible symbols, or something like that?