SMB vs rclone mount file operation compatibility

[ncw (Nick Craig-Wood)]

@Forum1877621

I was wondering why SMB seems to have much better (perfect?) file operation support compared to rclone mount without requiring a cache.

This is the correct answer

@Animosity022

I'd imagine trying to compare SMB which is a specific use case for sharing via Windows vs a rclone mount, which is fuse based, supports a plethora of backends in terms of numerous cloud storage based providers and comparing that to a local shared protocol on a LAN doesn't really seem like comparing apples.

I'll expand it though.

JDownloader requires the ability to write blocks in the middle of a file. This is something which isn't supported by most cloud providers. You can't upload in the middle of an s3 object or a drive file - you have to upload the whole file again. This is the problem --vfs-cache-mode writes or --vfs-cache-mode full solves. Rclone keeps a local copy on disk so that it can write to the middle of it and upload it when done.

However network file systems like SMB (also SFTP) do support writing in the middle of files. Rclone is not yet clever enough to know this

So it would be possible to add another Backend method WriteAt to the SMB backend and expand the VFS layer to use that to write to the middle of files without --vfs-cache-mode writes.

This is a moderately complicated thing to do though (anything to do with the VFS layer is complicated!) and would only benefit a small number of backends: local, smb, sftp, hdfs so we haven't done it yet.

This would also allow these backends to be destination for --multi-thread-streams which would also be useful.