Using rclone from local to local file, not cloud

Help and Support
1

hello and thanks for taking the time to read this,

i wanted to know if anybody uses rclone to backup from a windows server file system to another windows server using local file system and network share.
not backing up to the cloud at all.
i am currently using robocopy but robocopy cannot do a validate/check/hash the copied files.

if so, please share your experiences.

thanks,
david

2

Rclone can work fine for this even if that isn't really it's primary focus, but there are also a lot of other tools that can do very similar things when it comes to non-cloud. rsync is basically that - and sort of the underlying inspiration for rclone as I understand.

On Windows I also use Teracopy a lot when it comes to doing manual copies that must be verified. rclone and similar tools are better suited to scripting for automatically recurring transfers but can be a little cumbersome to use for one-off jobs if you don't usually work with commandline so much. If you just want a much more advanced GUI replacement for the windows explorer copy dialogue that can automatically doublecheck transfers and the like then I recommend looking into that. The free version has most of the important features in it so you can try it out and see.

I'm sure there are a dozen other good sync tools for local files too, but these are just the ones I am the most familiar with. Someone else can probably add a few more suggestions.

Ultimately what tool is best is down to what your workflow and needs are.

3

thanks, i know of and use a bunch of programs for copying over the years.
i totally trust secondcopy, cobain, and fastcopy.
rsync on windows can be a problem with windows access permissions and what not.

one the main reasons i use rclone is --backup-dir archive feature.
secondcopy can do that and check copied files.

so i was thinking of using rclone for non-cloud and wanted feedback from other rclone users.
thanks,

4

Well rclone can do anything to local it can do to a cloud, and it's quite flexible (filtering ect.), so I think it is a good option personally. At least when it comes to automatic scripts that do periodic backups and the like. I use rclone for local stuff too sometimes. Not because there isn't other software that can do just as good a job at that, but because rclone does all the basics I need and I am already familiar with it - so why not? :smiley:

5

I know that there are quite a few users on Windows that use rclone for this. Rclone will take care of your data, checking hashes after the copy and as you said has some features like --backup-dir.

In 1.49 we put quite a bit of effort into making local -> local copies as fast as possible too.

6

ncw,
thanks for the info.

i checked page Rclone 1.49 release and the only changes, that might affect an effect in performance was "Don't calculate any hashes by default"
were there additonal changes made to local file systems that were not documented on that webpage.
thanks,
david

7

The not calculating hashes was a big part of the speedup. There was also running hashes in parallel and running fadvise on the local file system which will stop an rsync transfer of big local files pushing everything out of your cache.

These are the most important speedups

  • Disable multi thread copy for local to local copies (Nick Craig-Wood)
  • Avoid polluting page cache when uploading local files to remote backends (Michał Matczuk)
  • Don’t calculate any hashes by default (Nick Craig-Wood)
  • Fadvise run syscall on a dedicated go routine (Michał Matczuk)
8
  • Disable multi thread copy for local to local copies (Nick Craig-Wood)

This doesn't seem to apply to local -> local encrypted, so I had to use --transfers 1. Is that intended?

9

local -> local encrypted won't use multi thread transfers anyway. If you want to disable them use --multi-thread-streams 0.

--transfers 1 will mean only one transfer runs at once which if you don't have many cores may be beneficial. If you do have lots of cores then I would have thought it will hurt overall performance.

10

thanks for the reply.
i am very paranoid about backups.
so the idea that i would have to depend on rclone for both local backups and cloud backups makes me nervous.
i use veeam software for system backups, which uses checksums.
for local sync, i use fastcopy, as it has checksum feature.
for compressed files, i use 7z, which has checksum feature

thanks,

11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.