Egnyte Backend with Hashes (SHA512)

[jansan (Jan Sandorski)]

Hi,

I have merged the newest rclone version v1.63.0-DEV with the fork from here GitHub - egnyte/rclone: "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files in order to get a backend for Egnyte.

The above worked like a charm, however...

Egnyte does show hashes via API, they are SHA512. I tried digging around in the rclone code a little bit but would need some guidance to allow e.g. --hash to work with lsjson.

At this stage, my only questions are:

1. Where do I start?
2. Is this considered very difficult or can this be achieved by a non-golang programmer?
3. How do I find the actual functions calling the API and listing the objects? I assume this is where we can start pulling the correct values for the hashes.

Any and all assistance is greatly appreciated. Regardless, I will continue my own research and will take the liberty of sharing here once found.

I am very well versed with Egnyte API and using it with Python. Have been doing this for many years.

Thanks in advance.

[ncw (Nick Craig-Wood)]

@jansan

Where do I start?

SHA256 is one of rclone's supported hash types so you don't need to add it.

What you need to do is make sure the Hashes function in the backend returns something like this

// Hashes returns the supported hash sets.
func (f *Fs) Hashes() hash.Set {
	return hash.Set(hash.SHA256)
}

Then implement the Hash function on object to return the hash as a lower case hex string. It does appear to do this already though, so maybe the Hashes above are disabled for a reason.

This function is wrong also, it should only return the hash if the incoming ty == hash.SHA256 otherwise it should return "" and an error.

// Hash returns the supported hash sets
func (o *Object) Hash(ctx context.Context, ty hash.Type) (string, error) {
	return o.sha512sum, nil
}

That should be enough to add support for hashes

It would be useful to see what command line you are using and what results you got.

Are Egnyte going to submit the backend at some point do you know?

[jansan (Jan Sandorski)]

This worked like a charm. I can now fully use --hash.

One note. I did add SHA512 to hash, as Egnyte does return SHA512. That way I can see which exactly is use across remotes.

I don't know whether Egnyte will submit this backend officially. I doubt that, to be honest.

However, I might, once we have fully completed all functionality and documentation.

Thank you Nick! As always, your help is invaluable!

[jansan (Jan Sandorski)]

One more question regarding the experimental hasher backend, any chance it will work with SHA512?

[ncw (Nick Craig-Wood)]

@jansan

This worked like a charm. I can now fully use --hash.

Great

@jansan

One note. I did add SHA512 to hash, as Egnyte does return SHA512. That way I can see which exactly is use across remotes.

A backend can have multiple hashes supported and that is useful sometimes.

@jansan

I don't know whether Egnyte will submit this backend officially. I doubt that, to be honest.

However, I might, once we have fully completed all functionality and documentation.

@jansan

One more question regarding the experimental hasher backend, any chance it will work with SHA512?

I think hasher will support any hashes supported by rclone. You can list these with.

$ rclone hashsum
Supported hashes are:
  * md5
  * sha1
  * whirlpool
  * crc32
  * sha256
  * dropbox
  * hidrive
  * mailru
  * quickxor

This doesn't include sha512 but it would be a simple patch to fs/hash to add it.