rclone is set to use an SSH/SFTP remote to transfer files to. Sometimes there is a need to copy a file in a different path in the same remote and I would like to avoid using ssh to copy the file around.
From the docs it seems like rclone will always try to use a server-side action if it can. I also tried to use --server-side-across-configs but it still ends up being downloaded and uploaded.
edit after comment #5: The below only makes sense if the src and dst remotes are the same - I was considering only my specific scenario
Since cp in sftp is not supported (or still new?), would it be possible to a) use sftp ln instead or b) have rclone send an ssh remote cp src dest, like it does with md5sum when it detects it has shell access.
The above could be done when --server-side-across-configs or some sftp specific flag is used.
Run the command 'rclone version' and share the full output of the command.
os/version: debian 10.13 (64 bit)
os/kernel: 4.19.0-25-amd64 (x86_64)
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg rclone copy /tmp remote:tmp)
with sftp, rclone does not support server-side copy.
tho, does support server-side move.
DEBUG : sftp://email@example.com:23/zork/: Shell type "unix" from config
DEBUG : sftp://firstname.lastname@example.org:23/zork/: Relative path resolved to "/home/zork"
DEBUG : sftp://email@example.com:23/zork/: Using root directory "/home/zork"
DEBUG : file01.ext: Need to transfer - File not found at Destination
INFO : file01.ext: Moved (server-side) to: file02.ext
As I mentioned I am aware that SFTP does not support copy.
I suggested some alternatives to that though I understand it is most likely a niche case so it might end up at the bottom of the feature list or not even at all for ncw
Well it is, you are right. My mistake was that I forgot that this is applicable only when the src and dst is the same sftp remote. If they are different, obviously we enter the voodoo land. Which makes it an even more niche or N/A case.
I'm not sure all sftp servers will support this and certainly not all file systems support hardlinks so it needs to fall back intelligently to a normal copy. I didn't implement that for the POC though!
Whether a hardlink is as good as copying the file is another question that probably needs addressing. Maybe the backend needs an option to explicitly enable this, eg --sftp-copy-is-hardlink or something like that.
It comes down to when you modify the file. So if you are taking immutable backups there is no problem. But if you append data to one of the hardlinks, you'll be appending it to both. Most editors take care to break hardlinks first if you edit hardlinked files, but not everything does.
It occurs to me that I could implement this flag for the local filesystem too. That would then make --copy-dest for instance make a zero sized fully populated backup.
I agree with the other peeps that it should have its own flag, as copy is not the same as hardlink.
Would like to chime in just that the documentation/help, as the flag would become quite long? should make it very clear that this flag has an effect only when the two remotes are the same otherwise it will be ignored and work as a normal copy.
I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.65
Here is the config item to enable it.
Set to enable server side copies using hardlinks.
The SFTP protocol does not define a copy command so normally server
side copies are not allowed with the sftp backend.
However the SFTP protocol does support hardlinking, and if you enable
this flag then the sftp backend will support server side copies. These
will be implemented by doing a hardlink from the source to the
Not all sftp servers support this.
Note that hardlinking two files together will use no additional space
as the source and the destination will be the same file.
This feature may be useful backups made with --copy-dest.