What is needed to create a new backend?

If I wanted to create a backend, what are the functions that need to be exposed? I've tried searching the docs, but I didn't spot anything. But maybe I'm looking in the wrong place :slight_smile:

Pointers welcome!

Take a look at the docs above and if you have any more questions I'll answer them :slight_smile:

Backends have lots of optional features - you can start by implementing the required set then fill out the optional features.

The problem is "read a 1500 line source file and then read another 1500 line source file" isn't really documentation.

Maybe what we need is a dummy or example backend that people can use as a template.

Or maybe I'm just being fussy and lazy :slight_smile:

:wink:

If you want to start top down then see the Fs interface and the Object interface - these are what a backend has to define.

Looking at the code for a backend will make more sense then.

There are lots of little details which the integration tests will check.

So the procedure I use for writing a backend is

  • copy one of the existing backends
  • alter to be my new backend in the most minimal way so it still compiles
  • get the login part to work
  • implement List
  • start running the integration tests and work through the failures until the backend is done
  • implement any optional methods the backend is capable of

I wouldn't start a backend from scratch as having the backends written all in the same style makes them much more maintainable.

It usually only takes an hour or two to get to running the integration tests and once you are there the integration tests guide you as to exactly what needs implementing.

Trouble with a dummy backend is that it isn't that useful!

The closest thing to that is the memory backend which is a bucket based backend - so if you want to study a backend in detail to see how they work, I'd start there. It also implements quite a few optional features, eg Copy.

However I would start as above to actually write a backend - there is all sorts of useful infrastructure you may need (eg the directory cache module, the rest library) which will be shown in the example you choose.

What kind of backend are you thinking about? I can help you pick a backend to start from.

I was just thinking back to an earlier discussion around a "program" backend which would act as a layer that could transform data (eg encrypt or decrypt, or whatever) data. It wouldn't be performant on lots of small files but may not be too bad on larger ones. So I took a look at crypt and chunker but these aren't so trivial.

I probably wouldn't have time to do anything about it, anyway; was just idle curiousity and then I hit the high barrier to entry!

Interesting idea, so a backend which pipes data in and out of an external program like gpg for example.

I can see partial reads (seeking) and knowing the lengths of the decrypted files might be a problem. You could possibly solve some of that with some metadata...

Let me know if you want more help!

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.