Add random bytes to the end of each file before encrypting and uploading

hi all,

Do you think it would be possible to add random bytes to each file before encrypting and uploading the file.

Υοu could make the argument that this is probably redundant given that the hash is dependent on the encryption which is in turn dependent on the users's selection (256,512,1024) for the rclone crypt remote. Having said all that, what are people's thoughts ?

example: if filename is Friends s01e02.mkv then it should be Friends s01e02_934&^G%.mkv which will then get crypted and uploaded

Alternatively, if anyone knows of any programs in Windows or linux that already do this then please share them.

What are you trying to accomplish by doing this? and it wouldn't be backward compatible.

Added evasiveness from hash detecting mechanisms. I'm clearly paranoid beyond what you may consider reasonable haha

For filenames, that's what the filename encryption and obfuscation routines do.

% rclone lsd mp3: | head -3
          -1 2018-06-16 21:34:23        -1 BAD
          -1 2018-06-16 21:34:22        -1 DONE
          -1 2018-06-16 21:34:30        -1 OLD

% rclone lsd Amazon:mp3 | head -3 
          -1 2018-06-16 22:10:18        -1 138.YUTMY
          -1 2018-06-16 21:34:23        -1 199.SRU
          -1 2018-06-16 21:34:30        -1 223.ebT

The cloud provider will just see the obfuscated/encrypted names ("138.YUTMY") and these values are based on the passwords you chose.

Similarly the data inside the file is uniquely encrypted based on the passwords you chose.

So both your filenames and your content are unique to you and won't trigger any hash matches.

Τhis makes sense to me. Can you please ELI5 in one sentence or fewer words what each of the 6 lines (2 sets of 3) refer to ? i.e. bad, done, old, 138.yutmy, sru,ebt

The lsd mp3: is how it looks to me using rclone to decrypt the filenames. The lsd Amazon:mp3 shows how it actually looks on the cloud server and so what the provider sees.

In my config I have:

[mp3]
type = crypt
remote = Amazon:mp3
filename_encryption = obfuscate
password = yeahyeah
password2 = sowhat

So we can see that the mp3: remote is just a folder inside the Amazon: remote.

that is not really an encrypted file name, just a simple ROT.

i do not think that there is any encryption going on with that filenames using obfuscation.
you would need to enable standard file encryption

but i could be wrong

Yes. The filename is "obfuscated". The hint is in the name. But it's enough to prevent pattern matching! For me, obfuscation is enough. But "standard" can be used if you want stronger encryption. That's why I wrote "obfuscated/encrypted" in my reply, above.

From the documentation (https://rclone.org/crypt/)

Obfuscation

This is a simple “rotate” of the filename, with each file having a rot distance based on the filename. We store the distance at the beginning of the filename. So a file called “hello” may become “53.jgnnq”

This is not a strong encryption of filenames, but it may stop automated scanning tools from picking up on filename patterns. As such it’s an intermediate between “off” and “standard”. The advantage is that it allows for longer path segment names.

There is a possibility with some unicode based filenames that the obfuscation is weak and may map lower case characters to upper case equivalents. You can not rely on this for strong protection.

  • file names very lightly obfuscated
  • file names can be longer than standard encryption
  • can use sub paths and copy single files
  • directory structure visible
  • identical files names will have identical uploaded names

Cloud storage systems have various limits on file name length and total path length which you are more likely to hit using “Standard” file name encryption. If you keep your file names to below 156 characters in length then you should be OK on all providers.

thank you for your detailed explanation. This is crystal clear to me now

yes, we agree,
Obfuscation does not encrypt the filenames using passwords.
just a simple ROT

It's not quite a simple ROT. The rot distance uses the password as a component of the calculation. If two people use different passwords then they will end up with two different obfuscated file names.

This makes it even harder to detect; you need to know the password to "unrot" the filename into a real name.

(I wrote that code).

1 Like

am i correct that there is not an obfuscate option for directories?
if true, why would that be?

Directories are obfuscated the same as files if you use the filename_encryption = obfuscate option.

Indeed, in the example above, "BAD", "DONE" and "OLD" are all directories and you can see they're obfuscated on the server.

i am able to enable obfuscate on filenames but not folders.

as per the website, Directory name encryption is true and false, no option for obfuscate.

so how do i get obfuscate for folders?

thanks

If you turn on filename_encryption = obfuscate then it obfuscates both files and directories.

% rclone ls Amazon:mp3 | head -20
     1583 114.yljvkl
     1793 128.mzIvHz_ZVX
     2074 172.tCrDst.BE8
      180 205.BJyBA_FJxDB.qbpq
      509 207.!!qdZcld
      892 254.yDAyG 
      713 31.AMTCP  
       95 56.AMLTCPR_RM_DJyA
      348 7.ARBzM   
     2664 95.rzorq_vzntr
      359 138.YUTMY/251.BGwxQ.AMFE
  3768266 38.JUTK/155.YxOOLK_hKFDEQP/66.J_cjBCn_Xo_JppAx.vy0
 51956684 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/172.IGprz56.rssp.LpK
 39680956 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/172.IGprz65.rssp.LpK
 44894252 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/173.JHqsA68.sttq.MqL
 42600492 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/173.JHqsA77.sttq.MqL
 36810812 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/173.JHqsA86.sttq.MqL
 39523324 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/174.KIrtB70.tuur.NrM
 35300460 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/174.KIrtB89.tuur.NrM
 39617436 223.hUikbj/233.Wbsjpvt/186.asqer_Xs_asqer_xeoi2/175.LJsuC82.uvvs.OsN

You can see the directories (eg "223.hUikbi") are obfuscated.

interesting,

  1. is this documented?
  2. i have never done it, but i could add that flag to my config file?

The config I provided (except for the passwords!!) is my live config that I used for this.

[mp3]
type = crypt
remote = Amazon:mp3
filename_encryption = obfuscate
password = yeahyeah
password2 = sowhat

All the output I've presented is real output as a result of that config.

how do i add a flag to a confile?

What flag? Those 6 lines are the complete definition of a remote.

1 Like

out of curiosity why do you use obfuscate and not standard encryption for file names ?