Sync Couldn't delete: object not found

Gillesvv · January 11, 2024, 11:05am

What is the problem you are having with rclone?

In addition to other backup strategies in our company we daily sync critical data to our S3 provider, it has been working perfectly for months but this week we are starting to see the same errors.

Couldn't delete: object not found

Our sync method is

local folder -> rclone compressed -> rclone encrypted -> s3 storage (95GB~)

I suppose this means that the file cannot be found on the remote S3 bucket?
Why throw an error for deletion if the file doesn't exists?
Does this mean that the file was never synced when it existed in the source?

Manually purging the files with rclone works but it is a time consuming effort, some suggestion in this forum after google-ing was to use the --ignore-errors flag.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.1
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-88-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.5
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OVH S3 Object store

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone --retries 1 -vvvv --ignore-errors --log-file=/tmp/rclone.debug.log3 /local/gitlab/git-data ovh_compressed:/sync/local/gitlab/git-data

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[ovh_backup_dev]
type = s3
provider = Other
access_key_id = XXX
secret_access_key = XXX
endpoint = https://s3.gra.io.cloud.ovh.net/
region = gra

[ovh_encrypted]
type = crypt
remote = ovh_backup_dev:rclone-critical
directory_name_encryption = false
password = XXX
filename_encryption = off

[ovh_compressed]
type = compress
remote = ovh_encrypted:vierwerf/
level = 6

A log from the command that you were trying to run with the `-vv` flag

...
2024/01/11 10:44:57 DEBUG : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/info/commit-graphs/graph-8f4a2b7acb0e9c5ed4743e73dac59018f8fc6a2a.graph: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/11 10:44:57 DEBUG : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/info/commit-graphs/graph-8f4a2b7acb0e9c5ed4743e73dac59018f8fc6a2a.graph: Unchanged skipping
2024/01/11 10:44:57 DEBUG : Compressed: ovh_compressed:/sync/local/gitlab/git-data: Waiting for transfers to finish
2024/01/11 10:44:57 DEBUG : Waiting for deletions to finish
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/ab7682b5440025a17ed65e58ecdc76826ddddc47: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/b3/f747449d1a3e15da10343f9c305d8fc0b8149d: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/48/2ae82f811c35c0c3026cc2170740652c78e3dd: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/fb/9e8a74014601c9c933aad90ce39441e29e6735: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/25c5e64304227b40cce1de130ce5b914e1ba777d: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/46/c70d3064ab378b6e5217ee349875faa654f9e0: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/14/ea4e6c42e794d8ddbd6dc3302bf96106c6e458: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/a6/4da47730efc730ef6fcbc1cb6aa7e5e7850b28: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/bd/59b44c33e0b30d7f3c11df962ffc0dae423f0d: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/fdd7dc8496f66f723567008ff577a68c3e35a451: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/482ae82f811c35c0c3026cc2170740652c78e3dd: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/a4/5c1b3e0a508e2254eb444b77b6dde50ca2632e: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/24/3c06d4ee6d2fd3fc4c3e80e17bc2904ab135f8: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/d8c888489c26469e99716063de0ac5faf4bb69d2: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/1c/c49aa4444832a09a23e1f2c9e5d01733939185: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/pack/pack-79096758e354237569202a804ed480f69bf20477.rev: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/bf/26a4b24e35a5ab6e90080a6c04b0ef0c782a4d: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/b2/5bc309d91e6a7923f56d36450ce60f7b8d233c: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/e3f50b8e4e6303f3d5d104e68f177a7eebd5834e: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/23695d811a354c5b85398a7d1a5f1bd32c947ecb: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/1b59de225a325975f5ff8846422a46e99cfe6c64: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/6ee6dba80f52d44e8da4647cae6913770f861876: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/ec/eb48961daff6b98b592a83a843e311939bcfd5: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/1a/e6d8a58f91ac3582a5cd2611c9cb92ee5220d3: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/a2/d0c5f3bd34d1c49840a9d7329414da75978ae6: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/8ef220961f8011465f830da3f013ddeae7e39aa7: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/05/a9b6f3ffd954adc366a2f112df7decdfbead5e: Couldn't delete: object not found
2024/01/11 10:44:57 ERROR : Attempt 15/15 failed with 28 errors and: failed to delete 27 files
2024/01/11 10:44:57 INFO  : 
Transferred:   	   43.217 KiB / 43.217 KiB, 100%, 1.373 KiB/s, ETA 0s
Errors:                28 (retrying may help)
Checks:              6215 / 6215, 100%
Deleted:               27 (files), 0 (dirs)
Transferred:           21 / 21, 100%
Elapsed time:        31.6s

2024/01/11 10:44:57 DEBUG : 63 go routines active
2024/01/11 10:44:57 Failed to sync with 28 errors: last error was: failed to delete 27 files

asdffdsa · January 11, 2024, 1:26pm

welcome to the forum,

rclone --retries 1 -vvvv --ignore-errors --log-file=/tmp/rclone.debug.log3 /local/gitlab/git-data ovh_compressed:/sync/local/gitlab/git-data
that does not look like a valid command, the command is missing?

best to run the command with --log-level=DEBUG
and post the top 30 lines of the debug log?

maybe you updated rclone and then that errors started?

Gillesvv · January 12, 2024, 8:28am

Mybad, it the command is sync what we use.

rclone sync --retries 1 -vvvv --ignore-errors --log-file=/tmp/rclone.debug.log3 /local/gitlab/git-data ovh_compressed:/sync/local/gitlab/git-data

rclone sync --retries 3 --log-level=DEBUG --ignore-errors --log-file=/tmp/rclone.debug.log4 /local/gitlab/git-data ovh_compressed:/sync/local/gitlab/git-data

2024/01/12 09:19:15 DEBUG : rclone: Version "v1.65.1" starting with parameters ["/bin/rclone" "sync" "--retries" "3" "--log-level=DEBUG" "--ignore-errors" "--log-file=/tmp/rclone.debug.log4" "/local/gitlab/git-data" "ovh_compressed:/sync/local/gitlab/git-data"]
2024/01/12 09:19:15 DEBUG : Creating backend with remote "/local/gitlab/git-data"
2024/01/12 09:19:15 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/01/12 09:19:15 DEBUG : Creating backend with remote "ovh_compressed:/sync/local/gitlab/git-data"
2024/01/12 09:19:15 DEBUG : Creating backend with remote "ovh_backup_dev:rclone-critical/vierwerf/sync/local/gitlab/git-data.json.bin"
2024/01/12 09:19:15 DEBUG : Resolving service "s3" region "gra"
2024/01/12 09:19:16 DEBUG : Creating backend with remote "ovh_backup_dev:rclone-critical/vierwerf/sync/local/gitlab/git-data.json"
2024/01/12 09:19:16 DEBUG : Resolving service "s3" region "gra"
2024/01/12 09:19:17 DEBUG : Creating backend with remote "ovh_backup_dev:rclone-critical/vierwerf/sync/local/gitlab/git-data.bin"
2024/01/12 09:19:17 DEBUG : Resolving service "s3" region "gra"
2024/01/12 09:19:17 DEBUG : Creating backend with remote "ovh_backup_dev:rclone-critical/vierwerf/sync/local/gitlab/git-data"
2024/01/12 09:19:17 DEBUG : Resolving service "s3" region "gra"
2024/01/12 09:19:17 DEBUG : repositories/+gitaly/state/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/latest: Need to transfer - File not found at Destination
2024/01/12 09:19:17 DEBUG : repositories/+gitaly/state/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/latest: Need to transfer - File not found at Destination
2024/01/12 09:19:17 DEBUG : repositories/.gitaly-metadata: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/.gitaly-metadata: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git/HEAD: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git/HEAD: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git/HEAD: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git/HEAD: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git/config: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git/config: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git/.gitaly-full-repack-timestamp: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4e/07/4e07408562bedb8b60ce05c1decfe3ad16b72230967de01f640b7e4729b49fce.git/.gitaly-full-repack-timestamp: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git/config: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git/config: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/HEAD: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/HEAD: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/config: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/config: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git/description: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git/description: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git/HEAD: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git/HEAD: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git/config: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git/config: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/HEAD: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/HEAD: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/+gitaly/state/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/latest: md5 = 2e49d179f98c2d72f479034b51630127 OK
2024/01/12 09:19:17 DEBUG : repositories/@snippets/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/description: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/description: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/config: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/config: Unchanged skipping
2024/01/12 09:19:17 DEBUG : repositories/@snippets/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git/refs/heads/main: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/01/12 09:19:17 DEBUG : repositories/@snippets/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git/refs/heads/main: Unchanged skipping

Until two days ago we were running 1.64.0, then I upgraded the binary to 1.65.1 to see if the errors would be resolved, but same result on both versions.

asdffdsa · January 12, 2024, 2:01pm

that last debug log looks ok, no errors, and file(s) were copied successfully.

Gillesvv · January 12, 2024, 2:11pm

Same log last lines.

2024/01/12 09:21:15 DEBUG : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git/objects/info/commit-graphs/graph-1d41790404d6783f50bb1cfdcbe70a485b0bb312.graph: Unchanged skipping
2024/01/12 09:21:15 DEBUG : Compressed: ovh_compressed:/sync/local/gitlab/git-data: Waiting for transfers to finish
2024/01/12 09:21:15 DEBUG : Waiting for deletions to finish
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/ec/eb48961daff6b98b592a83a843e311939bcfd5: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/46/c70d3064ab378b6e5217ee349875faa654f9e0: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/24/3c06d4ee6d2fd3fc4c3e80e17bc2904ab135f8: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/23695d811a354c5b85398a7d1a5f1bd32c947ecb: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/48/2ae82f811c35c0c3026cc2170740652c78e3dd: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/1c/c49aa4444832a09a23e1f2c9e5d01733939185: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/ab7682b5440025a17ed65e58ecdc76826ddddc47: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/14/ea4e6c42e794d8ddbd6dc3302bf96106c6e458: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/a2/d0c5f3bd34d1c49840a9d7329414da75978ae6: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/b2/5bc309d91e6a7923f56d36450ce60f7b8d233c: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/05/a9b6f3ffd954adc366a2f112df7decdfbead5e: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/6ee6dba80f52d44e8da4647cae6913770f861876: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/d8c888489c26469e99716063de0ac5faf4bb69d2: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/1b59de225a325975f5ff8846422a46e99cfe6c64: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/bd/59b44c33e0b30d7f3c11df962ffc0dae423f0d: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/pack/pack-79096758e354237569202a804ed480f69bf20477.rev: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/e3f50b8e4e6303f3d5d104e68f177a7eebd5834e: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/8ef220961f8011465f830da3f013ddeae7e39aa7: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/b3/f747449d1a3e15da10343f9c305d8fc0b8149d: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/a6/4da47730efc730ef6fcbc1cb6aa7e5e7850b28: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git/objects/30/53ea200a0fe088e264d8b9695cea6fbc609c2e: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/25c5e64304227b40cce1de130ce5b914e1ba777d: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/1a/e6d8a58f91ac3582a5cd2611c9cb92ee5220d3: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/refs/keep-around/fdd7dc8496f66f723567008ff577a68c3e35a451: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/objects/a4/5c1b3e0a508e2254eb444b77b
6dde50ca2632e: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/bf/26a4b24e35a5ab6e90080a6c04b0ef0c782a4d: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/4b/22/4b227777d4dd1fc61c6f884f48641d02b4d121d3fd328cb08b5531fcacdabf8a.git/objects/fb/9e8a74014601c9c933aad90ce39441e29e6735: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : repositories/@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git/refs/keep-around/482ae82f811c35c0c3026cc2170740652c78e3dd: Couldn't delete: object not found
2024/01/12 09:21:15 ERROR : Attempt 3/3 failed with 31 errors and: failed to delete 28 files
2024/01/12 09:21:15 INFO  : 
Transferred:       54.948 KiB / 54.948 KiB, 100%, 4 B/s, ETA 0s
Errors:                31 (retrying may help)
Checks:             15717 / 15717, 100%
Deleted:               84 (files), 0 (dirs)
Transferred:           26 / 26, 100%
Elapsed time:       2m0.3s

2024/01/12 09:21:15 DEBUG : 37 go routines active
2024/01/12 09:21:15 Failed to sync with 31 errors: last error was: failed to delete 28 files

asdffdsa · January 12, 2024, 2:27pm

i am not sure the exact issue. maybe an issue with experimental compress remote, maybe an issue with ovh support for s3 or something else.

the compress remote is experimental.
https://github.com/rclone/rclone/issues/6298
"I tested it by simulating a dry-run by wrapping an an existing remote without success"

95GB~ - that is a very small amount of data, do you really need the compress remote?
might test without it, confirm if that is the problem.

given the source is local, might compress using a local tool, then upload.
fwiw, that is what my backup script does, 7z the source files, then upload .7z
or compress each file, then upload each individual file.

would be great if the same file fails each time, then the log would be kept small.
rclone sync /path/to/file remote: -vv --dump=headers --retries=1 --low-level-retries=1

Gillesvv · January 15, 2024, 1:43pm

Our uplink is limited hence we want the compression enabled.

Paste bin log truncated timestamps because of pasebin's 512kb upload limit.

Gillesvv · January 18, 2024, 8:05am

Problems still persist, anyone?

Gillesvv · February 5, 2024, 7:47am

Sorry for bump, anyone?

system · March 6, 2024, 7:48am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.