Rclone serve for Duplicacy like restic API


#1

Hi all,

I am delighted to open this new topic (I am doing it in both rclone and Duplicacy forum) to request a future collaboration between two of the best tools all over the internet for backing up data which is not yet in place.

Basically this Feature request is to be able to create a serve a remote over a given protocol with Rclone to serve files for Duplicacy so we can perform backups directly similar as the restic integration and we can use duplicacy in those storage provides not supported yet.

Now I am backing up it with “rclone” mount option, but that makes all the files available in the server as a mount point (that can lead to mistakes due to incorrect deletions).

I have tested the restic rclone integration and it works like a charm, hence I am looking forward to see such a good piece of work for the two best backup tools (https://github.com/gilbertchen/benchmarking) I have the pleasure to use. Rclone and Duplicacy.

What do you think?


#2

Sounds like an interesting idea! Does Duplicacy have an API for backends defined already? I had a squint at the source code and it doesn’t look like it.

I do note that duplicacy supports webdav and rclone can serve it with rclone serve webdav that might be a way forwards that is more efficient than using a mount?

Can you see if that works? If it does then we can think about how to make a slicker integration.


#3

Hi Nick,

to be honest, I do not know, as said, I have opened in both forums to “open an official request” for collaboration :wink:

I am aware about the rclone server webdav, I already made a lot of attempts, however, I ended up opening a bug with duplicacy (please, take a look here https://forum.duplicacy.com/t/rclone-webdav-storage-path-test-does-not-exist/1821) as looks like they do now work well with the rclone webdav implementation (or rclone implementation does not work well with duplicacy webdav client :smiley:)

If at least webdav could work sharing files to rclone, it’s better than nothing. Of course is always better a “direct serve”.

Thank a lot for your reply and the time to consider such collaboration.


#4

OK well if we work on getting the webdav implementation to work that would be a good step forward.

The webdav protocol isn’t particularly well specified so there are lots of divergences in the implementation.

The webdav server rclone uses is https://godoc.org/golang.org/x/net/webdav

The restic implementation in terms of efficiency is pretty similar to webdav so I think if we can make that work it will work quite well.

Can you please make a new issue on github about the incompatibility. Can you put some simple reproduction instructions (assume we know nothing about duplicacy) and we can try to work out where the incompatibility lies.


#5

Hi Nick,

thank so much… here you can find it I am not very well with the subject text :wink:

I have included all the steps for the certs generation as looks like duplicacy does not work well without the autosigned off cert or without signed off certs and to speed up any test you want to do.

whatever thing else, you need to reproduce any step, let me know, I have the environment ready for any test.


#6

@ncw , more info about…

fixed a webdav compatibility issue with rclone and other bugs

Looks like gchen has helped a lot, and maybe that will make possible integrating duplicacy with “rclone serve webdav”.

Take a look to it.