Object Not Found Error

Suspected Bug
1

What is the problem you are having with rclone?

I have IceDrive mounted using rclone via webdav, I did run rclone sync -P old_drive: ice_drive: to copy all my files to the new ice_drive mount, it was around 200GB, mostly when well was getting some object not found errors, at the end failed uploads count to 71 - around 3GB.
When I try to run the sync command again it fails for the all of the files, getting object not found error for each of them, I tried to get one of the failed file and copy from the source location to the target localtion manually, by copy/paste, for following error from the log file:
File name is Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf

2021/12/06 13:43:29 DEBUG : Books/All/M4lware Analysis/Books/Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf: vfs cache: starting upload
2021/12/06 13:43:52 ERROR : Books/All/M4lware Analysis/Books/Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf: Failed to copy: object not found
2021/12/06 13:43:52 ERROR : Books/All/M4lware Analysis/Books/Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf: vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file from cache to remote: object not found

It will try after 10s, same thing happens and same again and again.

It was the exactly same file sync is failing, let's try another file, this one was synced well, I just tried to copy it from old_drive to a random location into the new one: vb.pdf

2021/12/06 13:54:07 DEBUG : Books/all/vb.pdf: vfs cache: starting upload
2021/12/06 13:54:10 INFO  : Books/all/vb.pdf: Copied (new)
2021/12/06 13:54:10 DEBUG : Books/all/vb.pdf: vfs cache: fingerprint now "354243"
2021/12/06 13:54:10 DEBUG : Books/all/vb.pdf: vfs cache: writeback object to VFS layer
2021/12/06 13:54:10 DEBUG : Books/all: Added virtual directory entry vAddFile: "vb.pdf"
2021/12/06 13:54:10 INFO  : Books/all/vb.pdf: vfs cache: upload succeeded try #1

It worked, so webdav works, no rate limit.

Let's try to the failed file to copy into anothor location in the new mount:

2021/12/06 13:59:17 DEBUG : all/Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf: vfs cache: starting upload
2021/12/06 13:59:37 ERROR : all/Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf: Failed to copy: object not found
2021/12/06 13:59:37 ERROR : all/Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf: vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file from cache to remote: object not found

Same, looks like the issue is with file.

File meta:

ExifTool Version Number         : 12.30
File Name                       : Michael Hale Ligh, Andrew Case, Jamie Levy, AAron Walters - The Art of Memory Forensics, Detecting Malware and Threats in Windows Linux and Mac Memory - 2014.pdf
Directory                       : .
File Size                       : 27 MiB
File Modification Date/Time     : 2021:11:07 00:51:43+00:00
File Access Date/Time           : 2021:11:07 00:51:43+00:00
File Inode Change Date/Time     : 2021:11:07 00:51:43+00:00
File Permissions                : -rw-r--r--
File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.6
Linearized                      : Yes
Page Layout                     : SinglePage
Page Mode                       : UseOutlines
Page Count                      : 914

What's possible reason that I can't copy the files from one drive to another?
Is they any details I need to provide?

What is your rclone version (output from rclone version)

rclone v1.57.0
- os/version: arch 21.2.0 (64 bit)
- os/kernel: 5.10-MANJARO (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.2
- go/linking: dynamic
- go/tags: none

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux - Manjaro - kernel 5.10

Which cloud storage system are you using? (eg Google Drive)

Source: Google Drive
Destination: IceDrive - webdav

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync -P --tpslimit 5 --transfers 1 gdrive_crypt: ice_drive_crypt:

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = REDACTED
client_secret = REDACTED
scope = drive
token = {"access_token":"REDACTED","expiry":"REDACTED"}
team_drive = 

[gdrive_crypt]
type = crypt
remote = gdrive:REDACTED
password = REDACTED

[ice_drive]
type = webdav
url = https://webdav.icedrive.io
vendor = other
user = REDACTED
pass = REDACTED

[ice_drive_crypt]
type = crypt
remote = ice_drive:REDACTED
password = REDACTED

A log from the command with the -vv flag

Already provided above
2

hi,

ice_drive_crypt should be ice_drive_crypt:
perhaps just a simple typo, but when dealing with a potential rclone bug, we have to make sure everything is 100% accurate.

3

Sorry, just missed it during copy. updated

4

ok, good,

  • if you can replicate the issue with rclone sync instead of rclone mount, that would be better.
  • would be most hepful to post/upload a full debug log.
5

I'm already using rclone sync

Unfortunately, I can't provide full debug log, lot's of personal details there, I can check for a specific file and specific error message :confused:

6

Is it something about the file name?

There doesn't look to be anything particularly unusual about the file name - maybe the ,?

I guess the other possibility is that it is too long at 161 characters. Since you are using crypt - I think that is most likely the problem as a 161 char file will expand to > 255 characters. You likely want to keep file names below 156 characters long.

7

It couldn't be filenames, because same is happening when I try to sync encrypted mounts directly without decryption (they have same passwords)

I'll try to reduce file name sizes and see if it helps

8

Reduced file name size and it worked!
Thanks Nick!

It was good for Google Drive, I think webdav have this limitation or maybe IceDrive itself

9

I don't think there is a limit in the webdav protocol so it is probably IceDrive. 255 chars is a common limit - Google Drive are exceptional in allowing 8,000 chars!

1 Like
10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.