Rclone with Backblaze B2 - duplicate (mostly empty) files

[harry66]

What is the problem you are having with rclone?

rclone sync created hundred or more (mostly empty) duplicate files in my B2 buckets
e.g.

3x7 label sheet.doc (hidden)  0 bytes 12/13/2021 09:34
3x7 label sheet.doc              30.2 KB  10/07/2018 11:27

The 2nd older files were already there - backed up from my previous NAS drive.
Recently, I copied the files locally to a new drive and NAS system and the timestamps changed - so I figured I would use the --checksum option to avoid duplication - based on timestamp.

But after I ran the rclone command (See below) all these (mostly empty) duplicate files appeared - but not for every file - just most. Some files are not duplicated.

I also tried dedupe option command but that only seems to delete other genuine duplicates I may have - not the above newly created files.

What is your rclone version (output from rclone version)

v1.57.0

Which cloud storage system are you using? (eg Google Drive)

Backblaze b2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync -P --checksum --exclude-if-present .ignore /mnt/media/Data BackB2:MyBucketName

The rclone config contents with secrets removed.

[BackB2]
type = b2
account = <removed>
key = <removed>

A log from the command with the -vv flag

So I have not rerun the command again as I don't want to create even more duplicates and I am not even sure it will have the same effect...Any ideas without having to get the verbose output?

[asdffdsa (jojothehumanmonkey)]

hello and welcome to the forum,

• as i understand it, b2 has a concept of hidden files.
  perhaps something to do with b2 versioning?
  https://rclone.org/b2/#versions

• have you searched the forum, as there are a number of posts about that.
  for example,
  https://forum.rclone.org/t/retrieving-hidden-files-from-backblaze-b2/14475

[harry66]

Hi - thanks for the reply.
I previously used B2 with some sync software on my nas so was not familiar with the file versioning stuff.
Have reached out again to B2 and checked my B2 bucket settings.

[jwink3101 (Justin Winokur)]

I feel like a debug log would be pretty helpful here. Could you do one or both of the following:

• Do it with --dry-run. That will help a little but at least
• Do a sync on a subdirectory?

Also, when you display the files, what utility is that? It may be showing a lot of hidden files and making things confusing.

[harry66]

Thanks, I also tried --dry-run with -vv did not yield anything useful - but will check again.
I really think @asdffdsa is on the right path - it something B2 is doing - not Rclone??