Rclone with Backblaze B2: where is the temporary directory in Mac OS X?

Help and Support
1

Hello there,

New user here!

Great software! Been using rclone with Backblaze B2 on Mac OS X.

I was going through the documentation here. It mentions:

When using B2 with crypt files are encrypted into a temporary location and streamed from there. This is required to calculate the encrypted file’s checksum before beginning the upload. On Windows the %TMPDIR% environment variable is used as the temporary location. If the file requires chunking, both the chunking and encryption will take place in memory.

Since I am on a Mac OS X, where are the temporary files stored for streaming purposes? Also, are these files deleted automatically after the sync operation has completed?

Thank you for the replies! :slight_smile:

2

Actually I think that documentation is out of date now that B2 allow streaming uploads. This was implemented in rclone in v1.38. The b2 backend will never spool files to disk now!

There is a slight drawback to this - see this issue for details.

I’m pretty sure it is /tmp on OS X and yes those files are deleted automatically. However they aren’t made any more as I said :wink:

I’ll fix the docs.

3

Thanks for the reponse Nick.

I read about the drawback in the link you gave. Just some follow ups regarding it:

What are the implications to this?
Why is this only a problem with large files and not smaller ones?
What is considered a ‘large’ file in terms of size?

Also appreciate if you could give your opinion on this thread.

Thanks for the help! Cheers!

4

Here is what I wrote in the docs.

The implication is that large crypted files won’t have checksums. So you can’t use cryptcheck on them.

It is a problem with large files due to the upload procedure used by B2 which may change in the future.

Large files are those uploaded in chunks, ie those greater than --b2-upload-cutoff which is 190M by default, but can be changed.

I’ll reply on that thread in a bit!

SHA1 checksums

The SHA1 checksums of the files are checked on upload and download and
will be used in the syncing process.

Large files (bigger than the limit in --b2-upload-cutoff) which are
uploaded in chunks will store their SHA1 on the object as
X-Bz-Info-large_file_sha1 as recommended by Backblaze.

For a large file to be uploaded with an SHA1 checksum, the source
needs to support SHA1 checksums. The local disk supports SHA1
checksums so large file transfers from local disk will have an SHA1.
See the overview for exactly which remotes
support SHA1.

Sources which don’t support SHA1, in particular crypt will upload
large files without SHA1 checksums. This may be fixed in the future
(see #1767).

Files sizes below --b2-upload-cutoff will always have an SHA1
regardless of the source.

5

Thank you! That makes a lot of sense.

Most of my files are below the b2-cutoff so I think I should be good.

Cheers!