Can the new hasher backend be shared amongst machines

Help and Support
1

What is the problem you are having with rclone?

I want to keep the new hasher kv store in sync amongst multiple computers. What are the downsides? From my reading, the issue will be with local remotes but for others (like remote crypts), it would be good to have them.

What is your rclone version (output from rclone version)

rclone v1.57.0
- os/version: darwin 10.15.7 (64 bit)
- os/kernel: 19.6.0 (x86_64)
- os/type: darwin
- os/arch: amd64
- go/version: go1.17.2
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Hasher

The command you were trying to run (eg rclone copy /tmp remote:tmp)

N/A

The rclone config contents with secrets removed.

[myhasher]
type = hasher
remote = anyremote:

A log from the command with the -vv flag

N/A

2

Also, just an aside, rclone makes it hard to find the caches. The noted place is right for linux only; not macOS (and presumably Windows). I just did a serve with vfs cache and -vv to find it

3

Latest version has introduced command config paths to reveal the local files/folders that rclone relies on.

Also rclone's common hash dir is now documented here, perhaps that also came with latest version - don't remember.

Do you mean documentation is wrong? Where? Edit: You probably mean the section at the end of the hasher docs? It does refer to "rclone cache directory", which is the one I linked to above, but I guess it would be better if it linked to it directly.

4

That's nice that you can set it. It does mean you have to be careful if serving a hasher remote with a custom --cache-dir which then negates the hash cache. Not the most common case but also reasonable to expect.

Yes, it was in the hasher. And it was always kind of hard to find for VFS caches (though I haven't looked with any updated 1.57 docs).

One note: The VFS has a DEBUG with the cache but the hasher doesn't print that

Any ideas on the original question? Can I copy the kv store between computers?

5

Sorry, if you are lucky @ivandeex stops by and gives you the answer! :wink:

6

If two machines have the same os/arch as per rclone version, and base remotes have the same name, you can just transfer bolt files from the hasher cache dir.

Generally you can export cache to Sum file on one machine (it's gonna be fast thanks to cache):
rclone hashsum sha1 my-hasher: --output-file my-sum.txt

Transfer it to the other machine and import:
rclone backend import my-hasher: sha1 my-sum.txt

Repeat the process for all required/supported hash types.
https://rclone.org/hasher/#pre-seed-from-a-sum-file

1 Like
7

Thanks. I saw those instructions but didn't think about it for that. I will play with doing that. I don't really have a need but it seems like I may as well wrap all of my crypted backends in the hasher just so I have it. Certainly not urgent though.

8

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.