Use raspberry pi as cache for syncing local files to remote

What is the problem you are having with rclone?

I don't have a Problem, I want to improve my backup routine

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2
-os/type: windows

Which cloud storage system are you using? (eg Google Drive)

remote A is wasabi and remote B is not decided jet, I'm open for everything, that can interact with my raspberry pi (raspbian) in my local network

The command you were trying to run (eg rclone copy /tmp remote:tmp)

At the moment I run

rclone sync /path/to/localfolder wasabi:localfolder

But since sometimes I have created 20GB-30GB of new files and my upload speed is not that fast, I have to let my pc run longer than I want to finish the upload. My idea to optimise this is as follows. I want to copy the files that have changed to my remote B on my raspberry pi. Since it is in my local network this is fast even for a larger amount of data. Than since my raspberry pi is always running it has enough time to upload everything. At the same time my pc should delete the files on remote A, which are not in my local folder any more. At the end the state on remote A should be as if I had just run the command above.
Is this possible to implement using rclone or maybe with help of some third party software and rclone.
Here are my ideas so far.
The easiest solution would be to sync everything to a remote on my raspberry pi and than sync this remote to wasabi. But since I have only 100GB available on my raspberry pi and over 1TB in my localfolder, I can't copy everything to the raspberry pi and have only space for the changed data.
An other idea is to use syncthing and sync my local folder to the mounted remote A with cache enabled. Then syncthing can sync all 30GB and this would be cached and when I shut down my pc, the rasperry pi keeps uploading the data. Is this possible?

Do you have any other ideas?

hi,
as for syncthing, that has been discussed a number of times in the forum.
https://forum.rclone.org/t/s3-crypt-settings-to-play-nicely-with-lots-of-scanning-syncthing/33733

that is what rclone sync does.

fwiw,a 1TiB hard drive is under $50.00

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.