State of the zip-backend

Hey there,

i found different conflicting information regarding this here in the forum.

Is it at the moment possible to work with zip files like that:

rclone copy /path/to/files remote:file.zip

or like that

rclone lsf remote:file.zip

as laid out in Limit file amount in remote - #6 by ncw

And how does the backend work? Does it "browse" the zip-file without loading it completely to RAM/cache?

How do i integrate this backend into my rclone version? Do i need to recompile it?
Since this is from 2022 would it work with the recent rclone-code?
Or was the zip-backend replaced by the "compress"-backend?

I am bit skeptical because in the documentation of compress it says it is best used with many large compressable files. I actually need something to work with many small compressable files (over 10.000 per folder) and since the contents of this zips wouldn't change i think the "zip-backend" would be a better option?

all those backends are experimental, hardly any posts in the forum.
imho, should not trust it in production unless you test it for yourself.

my backup scripts creates the .zip on local and then rclone move file.zip remote:
the advantages, password protected .zip file, can test the .zip before upload.

there is no zip backend...

...at least not in any released version. The referenced one is a prototype that hasn't been released and is branched off a really old version.

So you can do

rclone copy /path/to/files remote:file.zip

but it will create a folder, not a zip file, called file.zip.

I think the confusion is over what is zip vs what is the compress backend. Let's look at the files. This is not exact but close enough for all intents and purposes.

  • zip file: Container file of many smaller files. Commonly, each file is compressed with gzip compression
  • compressed gz file: A single file, usually with the name file.ext.gz that is compressed with gzip compression.

You didn't ask but to continue

  • tar file: Container file. No compression
  • tar.gz file: Container file that is then compressed. Often compresses better since it compresses all of the files at once, as opposed to each alone. But can't easily extract a single file.

The compression backend makes compressed gz files. There is no (production) zip backend. In fact, it would be very difficult to make one that is writable since zip files need to be rewritten when modified (for all intents and purposes). This was also in that forum post you linked.

For me, A read-only one would be great for mount since I've been unhappy with other zip-mount tools. But I guess not enough people seemed interested for @ncw to merge it in. I understand the decision though

1 Like

For me, A read-only one would be great for mount since I've been unhappy with other zip-mount tools.

This should be really overthought.
I see also big value in such a system. Even if it is only read only.
There is a lot of storage-backends that have filelimits and such a system would be very beneficial. Especially when the zip is browsable without downloading it fully in cache beforehand.
Which afaik should be possible.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.