Errors when tagging JPEGs

Suspected Bug
1

(First off, I only just started using rclone yesterday just after I signed up for BackBlaze B2 storage for my FreeNAS backups and it’s simply amazing… that it’s so cross-platform and feature rich, you have my thanks!)

Here’s what I did to reproduce this error (I’ve been testing with various options and repeating them, in fact the very last test I did showed that it apparently wasn’t ALL files without tags that I tagged that cause the problem, but so far it’s been nearly all of them (tested tagging non-tagged files probably 20 times now):

  1. Mounted drive, deleted old test files (deletes are very slow as well, not for 4 files, but in general)
  2. Copied 2 tagged files to Testing directory
  3. Added tag to each, one after the other
  4. Copied 2 non-tagged files to Testing directory
  5. Tagged one of the files successfully (no previous tags on the file)
  6. Tried tagging the other and got “Invalid file handle.” from windows with a Try Again/Skip option, I hit try again once, then skip when it reappeared
  7. I closed everything and hit control-c to stop the rclone mount.

https://paste.ee/p/YzB8F

Configuration:
(S: is a SMB share from a FreeNAS server, PC is Windows 10)

[Local]
type = local
nounc = false

[Fruits]
type = crypt
remote = Local:S:\Users\myuser\Fruits
filename_encryption = standard
directory_name_encryption = true
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***

For debugging, I used: rclone mount -v --dump-headers --debug-fuse --log-file rclone.log --vfs-cache-mode=minimal Fruits: Y:

Please let me know if I can do anything else to test with.

2

I suspect you need to use --vfs-cache-mode writes - the minimal mode is incompatible with some normal file system operations.

Can you give that a go and see if that helps?

3

I originally had tried writes and had the same error. Here is a log with the mount and then just updating the file tag (Windows reported the error, I hit retry once):

https://paste.ee/p/tYTxQ

I also tried with full vfs-cache with the same results.

4

There are less errors in the second log… however this one remains

2018/05/29 18:59:04 ERROR : photo_2018-05-08_12-59-29.jpg: Dir.Rename error: can't move object - incompatible remotes
2018/05/29 18:59:04 ERROR : IO error: can't move object - incompatible remotes

What apparently caused that was this

rclone[TID=3608]: FFFF9A82DFE9D2E0: >>SetInformation [Rename] "\photo_2018-05-08_12-59-29.jpg", 0000000000000000:0000000003B6CE70, NewFileName="\photo_2018-05-08_12-59-29.jpg~RF31bba0.TMP", AccessToken=00000000000003CC[PID=1e88]

It would be helpful to me to see a log of this happening with -vv and without --debug-fuse (it is a bit noisy!).

Can you replicate the error just by renaming things in a cmd window?

5

I did a rename within the CMD window in this debug without fuse-debug, with -vv and vfs-cache mode set to writes. The rename was successful with the rename command. I then tried adding a tag via explorer again to illustrate the difference in behavior.

Thanks for all your help!

https://paste.ee/p/D1hBF

6

Thanks for testing that.

This error sequence looks different…

The first error is this

2018/05/30 15:06:25 ERROR : photo_2018-05-08_12-59-29-new.jpg~RF4832180.TMP: corrupted on transfer: sizes differ 0 vs 172178
2018/05/30 15:06:25 INFO  : photo_2018-05-08_12-59-29-new.jpg~RF4832180.TMP: Removing failed copy

Which probably explains the second error (though rclone should probably have closed the file anyway)

2018/05/30 15:06:25 ERROR : ~hoto_2018-05-08_12-59-29-new.tmp: Dir.Rename cant rename open file

What exactly does it mean “add a tag via explorer” - I’m not familiar with that. Is it as described here?

Can you make an rclone mount which is just a local directory and see if the problem still exists?

rclone -vv --vfs-cache-mode writes mount C:\somedir Y:

I have access to a Windows server 2016 VM which I think is basically Windows 10 for servers so I’ll see if I can replicate the problem there if you can confirm exactly what you are doing to add a tag.

7

I tried locally with the same end result:
https://paste.ee/p/GeEE6

And yes, the tagging is as described in that article. It allows a platform independent method of adding keywords to image files for easy search (ie. “Bank statements”; 2017; Wells Fargo) It modifies the EXIF info of the file (or the equivalent on some other files (IPTC I think)).

8

I feel silly… I got it to work. I tried the beta version and it works there.

Thanks again for your amazing work and help with this. After payday I’ll be making a donation (this was the plan anyways, regardless of help with this issue), maybe you can take the wife out for dinner. ^^

Here’s the log output, FYI.
https://paste.ee/p/7hsnq

9

Great! I should have suggested that :blush:

The problem is around renaming open files for which a fix went in recently so I really should have thought of that.

:smile: