Some Questions regarding: checksum, crypt, mount, versioning [Jottacloud]

Help and Support
1

What is the problem you are having with rclone?

no checksum when using crypt,
as a workaround I mount with rclone mount rclone-crypt: /to/path
and use rsync --checksum --progress /source /to/path/ but than I lose file versioning / file history,

Any chance (or ideas how) to keep

  • my files encrypted in the jottacloud,
  • checked with checksum on transfer
  • maintain versioning / file history to be able to restore previous file versions.

What is your rclone version (output from rclone version)

rclone v1.52.0
os/arch: linux/amd64
go version: go1.14.3

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Lubuntu 16.04

Which cloud storage system are you using? (eg Google Drive)

Jottacloud

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Mounting the crypted volume:
rclone mount rclone-crypt:jottatest mount-test

Syncing files with checksum enabled:
rsync --checksum --progress -r jottatest/ mount-test/

The rclone config contents with secrets removed.

[rclone-crypt]
type = crypt
remote = acloud_jotta:cryp-dir
filename_encryption = standard
directory_name_encryption = true
password = BEEP

A log from the command with the -vv flag

Results in successfull transfer to the jottacloud in encrypted format but without versioning.

sending incremental file list
1_file.txt
             11 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=3/5)
rsync: rename "[...]mount-test/.1_file.txt.cgkKFH" -> "1_file.txt": Input/output error (5)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]

2020/06/11 15:58:50 ERROR : 1_file.txt: File.Rename error: couldn't move file: error 404: no.jotta.backup.errors.NoSuchFileException: Source file /5d2ee490149c9601334818c5/Jotta/Archive/cryp-dir/93hsgm0mrj5ksfbjiv176f4njg/vun0t29dvvm9k2779pi5jt6l6q1e6mjr6gqkpckivrhece9n4fh0 does not exist (Not Found)
2

I would just use rclone copy/sync and run a rclone cryptcheck if you want to validate the checksums.

Layering rscync on top of a mount feels like that creates a different set of issues as uploading is generally better not using a mount.

As for versioning, I'm not sure how that works or does not work in jottacloud.

3

you can read about checksum with crypt
https://rclone.org/commands/rclone_cryptcheck/

and for versioning, you might take a look at
https://rclone.org/docs/#backup-dir-dir

4

@Animosity022
@asdffdsa

Thank you for your advises!

Versioning / file history:
When I refered to versioning I mean the file-versioning / file-history made by Jottacloud itself and not implementing my own file versioning in the cloud. But I like to see that by using the simple --backup-dir remote:old parameter I should be able to use both ways. My own history or the one delivered by Jottacloud. My main point why using a cloud as backup storage is to prevent dataloss even when a malicious user with root access tries to delete or overwrite any file local or in the cloud.

Compare files by hash:
By only using the rclone copy command rclone will compare modification time size or MD5sum.
I want to force rclone to always compare hash / checksum and not to rely on size nor on modification time. But when forcing rclone to do the checksum with rclone copy --checksum it will ignore modified files.

For example: echo "one_line > 1_file.txt equals in size with echo "line_one" > 1_file.txt
So when comparing 1_file.txt it will differ in checksum and perhaps in modification time but not in size. Of course this is a rare case, but I want to make sure when doing backups not miss any file.

I don't understand how rclone cryptcheck can help me with this. Is it possible to automatically sync files that missmatch in their hashvalue?

echo "line_one" > jottatest/1_file.txt
rclone cryptcheck jottatest/ rclone-crypt:jottatest --progress -vv
2020/06/11 19:34:30 DEBUG : rclone: Version "v1.52.0" starting with parameters ["rclone" "cryptcheck" "jottatest/" "rclone-crypt:jottatest" "--progress" "-vv"]
2020/06/11 19:34:30 DEBUG : Using config file from "/home/alex/.config/rclone/rclone.conf"
2020/06/11 19:34:30 DEBUG : fs cache: renaming cache item "jottatest/" to be canonical "/mnt/jottatest"
2020-06-11 19:34:32 INFO  : Using MD5 for hash comparisons
2020-06-11 19:34:32 DEBUG : Encrypted drive 'rclone-crypt:jottatest': Waiting for checks to finish
2020-06-11 19:34:32 ERROR : 1_file.txt: hashes differ (rclone-crypt:jottatest) "bce46a5762a24732776cab64fae69a39" vs (local:/jottatest) "5ed074bb161cc066dfe93119b74b7a3f"
2020-06-11 19:34:32 DEBUG : 2_file.txt: OK
2020-06-11 19:34:32 DEBUG : subfolder/1_file.txt: OK
2020-06-11 19:34:32 NOTICE: Encrypted drive 'rclone-crypt:jottatest': 1 differences found
2020-06-11 19:34:32 NOTICE: Encrypted drive 'rclone-crypt:jottatest': 2 matching files
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 2 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         0.5s
2020/06/11 19:34:32 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 2 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         0.5s
5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.