Random "Failed to read metadata: object not found"

Help and Support
1

What is the problem you are having with rclone?

Performing rclone on an object storage in Scaleway is randomly showing NOTICE messages with content "Failed to read metadata: object not found".
However, the files for which the NOTICE message is shown are in the bucket.

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0
- os/version: opensuse-tumbleweed (64 bit)
- os/kernel: 6.8.8-1-default (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.8
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Scaleway object storage with storage class GLACIER.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy /tmp/data/Vault/ crypt-repo:/rhaidiz

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[remote-sw-paris]
type = s3
provider = Scaleway
access_key_id = XXX
secret_access_key = XXX
region = fr-par
endpoint = s3.fr-par.scw.cloud
acl = private
storage_class = GLACIER

[crypt-repo]
type = crypt
remote = remote-sw-paris:backup
filename_encryption = off
directory_name_encryption = false
password = XXX

A log from the command that you were trying to run with the -vv flag

This is an example of the log messages, the file in the logs exist in the bucket but I get this NOTICE.

2024/05/20 23:25:30 NOTICE: workspace/AndroidProjects/nfcwriter/app/build/intermediates/classes/release/com/worx/nfcwriter/R.class.bin: Failed to read metadata: object not found
2024/05/20 23:30:28 NOTICE: workspace/AndroidProjects/nfcwriter/app/build/intermediates/incremental/mergeReleaseShaders/merger.xml.bin: Failed to read metadata: object not found
2

welcome to the forum,

each time you run rclone, do the same files generate the message?

3

Nope, it seems to randomly show different files with the same message.

4

i ran into a similar issue a few years ago, tho, no idea if this is true in your case.

some S3 providers have issues with eventual consistency.
and maybe the issue is made worse using GLACIER

Each time rclone is run, 1/3 fails, 2/3 succeeds

5

I though about something like that, which would be quite a problem since I was planning on using rclone to backup my NAS :sweat_smile: .

6

recently, there have been a lot of changes in how rclone handles metadata.
i would test the last stable, v1.56.2

7

I'm on v1.66.0.

8

i am suggesting to test the last stable, v1.56.2

9

I'm a little confused on the versioning here :thinking: .
Anyway, as suggested I downloaded v1.56.2.

rclone v1.56.2
- os/version: opensuse-tumbleweed (64 bit)
- os/kernel: 6.8.8-1-default (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.8
- go/linking: static
- go/tags: none

But it looks like I'm getting the same issue. This is just one line, I've got other files as well, different form the previous run.

2024/05/21 20:19:21 NOTICE: workspace/poisontap/js/ajax.googleapis.com__ajax__libs__angular_material__0.11.2__angular-material.min.js.bin: Failed to read metadata: object not found
10

ok, i would test a simpler setup, without crypt, without GLACIER

11

Running it now, I'll keep you posted on the result (might take a while since the folder is roughly ~100GB and I'm re-uploading everything).
I'm still using v1.56.2.

12

Ok I've got some update. I started 3 rclone with v1.56.2 on 3 different destination folder and all from the same source folder.

  1. One with STANDARD object storage and no encryption.
  2. One with GLACIER object storage and no encryption.
  3. One with GLACIER object storage and encryption.

For (1) I get a few error messages such as these:

2024/05/22 01:52:48 ERROR : Foto&Video/[PATH]/IMG_0112.JPG: Failed to copy: object not found
2024/05/22 02:59:06 ERROR : School/[PATH]/links.pyc: Failed to copy: object not found
2024/05/22 08:30:25 ERROR : Attempt 1/3 failed with 2 errors and: object not found
2024/05/22 08:37:58 NOTICE: School/[PATH]/SeNGeMa-Tool.graffle: Failed to read metadata: object not found

However, all files are available from the source and all files seems to be available in the destination (rclone is still in progress by the way).

For (2) and (3) we have a similar situation with the only difference being that it has failed on many more files and, surprisingly, both (2) and (3) have failed on the same number of files (~17) but different files, not on the same files. Also in these cases the files appear to be on both the source and the destination and rclone is still in progress.

I'm quite not sure what is going on there :thinking: .

13

Another update. The test with STADARD object storage and no encryption has finished, so I run rclone again expecting no output. But I just got another NOTICE message with Failed to read metadata: object not found for a random file.

14

Ok I finally managed to solve this.
The issue was on the backend of Scaleway, which was returning a 404 that was throwing off rclone. I opened a ticket with them and they fixed the issue, great response :slight_smile: .

15

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.