Chunker upload to webdav error

Help and Support
1

What is the problem you are having with rclone?

when I upload chunker file to webdav server (Alist), got error.

What is your rclone version (output from rclone version)

root@debian:~# rclone -V
rclone v1.67.0

  • os/version: debian 11.10 (64 bit)
  • os/kernel: 5.10.0-31-amd64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22.4
  • go/linking: static
  • go/tags: none

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Debian12 64bit

Which cloud storage system are you using? (eg Google Drive)

webdav(by Alist) -> 123pan(www.123pan.com)

Test 1:
Config rclone with chunker storage, upload file to webdav server, got error.

[webdav]
type = webdav
url = http://xxxxx:5244/dav/

[chunker]
type = chunker
remote = webdav:/123pan/test
chunk_size =3800M
hash_type = sha1
name_format = *.#

root@debian:~# rclone copy test10.txt chunker:/ -vv
2024/08/11 16:56:49 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test10.txt" "chunker:/" "-vv"]
2024/08/11 16:56:49 DEBUG : Creating backend with remote "test10.txt"
2024/08/11 16:56:49 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 16:56:49 DEBUG : fs cache: adding new entry for parent of "test10.txt", "/root"
2024/08/11 16:56:49 DEBUG : Creating backend with remote "chunker:/"
2024/08/11 16:56:49 DEBUG : Creating backend with remote "webdav:/123pan/test"
2024/08/11 16:56:49 DEBUG : found headers: 
2024/08/11 16:56:49 DEBUG : fs cache: renaming cache item "webdav:/123pan/test" to be canonical "webdav:123pan/test"
2024/08/11 16:56:49 DEBUG : Reset feature "ListR"
2024/08/11 16:56:49 DEBUG : test10.txt: Need to transfer - File not found at Destination
2024/08/11 16:56:49 DEBUG : test10.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 16:56:50 DEBUG : test10.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 16:56:53 ERROR : test10.txt.1_1vq7jj: Couldn't move: copy NewObject failed: object not found
2024/08/11 16:56:53 ERROR : test10.txt.1_1vq7jj: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 16:56:53 ERROR : test10.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 16:56:53 ERROR : Attempt 1/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 16:56:53 DEBUG : test10.txt: Need to transfer - File not found at Destination
2024/08/11 16:56:53 DEBUG : test10.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 16:56:53 DEBUG : test10.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 16:56:54 ERROR : test10.txt.1_1vqbdf: Couldn't move: copy NewObject failed: object not found
2024/08/11 16:56:54 ERROR : test10.txt.1_1vqbdf: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 16:56:54 ERROR : test10.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 16:56:54 ERROR : Attempt 2/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 16:56:54 DEBUG : test10.txt: Need to transfer - File not found at Destination
2024/08/11 16:56:54 DEBUG : test10.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 16:56:55 DEBUG : test10.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 16:56:56 ERROR : test10.txt.1_1vqcq8: Couldn't move: copy NewObject failed: object not found
2024/08/11 16:56:56 ERROR : test10.txt.1_1vqcq8: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 16:56:56 ERROR : test10.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 16:56:56 ERROR : Attempt 3/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 16:56:56 INFO  : 
Transferred:   	         33 B / 33 B, 100%, 5 B/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         6.6s

2024/08/11 16:56:56 DEBUG : 8 go routines active
2024/08/11 16:56:56 Failed to copy: copy NewObject failed: object not found

Test 2:
directly upload file to webdav server to the same folder , no error.

[webdav]
type = webdav
url = http://xxxxx:5244/dav/

root@debian:~# rclone copy test13.txt webdav:/123pan/test/ -vv
2024/08/11 16:59:23 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test13.txt" "webdav:/123pan/test/" "-vv"]
2024/08/11 16:59:23 DEBUG : Creating backend with remote "test13.txt"
2024/08/11 16:59:23 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 16:59:23 DEBUG : fs cache: adding new entry for parent of "test13.txt", "/root"
2024/08/11 16:59:23 DEBUG : Creating backend with remote "webdav:/123pan/test/"
2024/08/11 16:59:23 DEBUG : found headers: 
2024/08/11 16:59:23 DEBUG : fs cache: renaming cache item "webdav:/123pan/test/" to be canonical "webdav:123pan/test"
2024/08/11 16:59:23 DEBUG : test13.txt: Need to transfer - File not found at Destination
2024/08/11 16:59:23 DEBUG : test13.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 16:59:24 INFO  : test13.txt: Copied (new)
2024/08/11 16:59:24 INFO  : 
Transferred:   	         11 B / 11 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.7s

2024/08/11 16:59:24 DEBUG : 8 go routines active

Test 3:
Config rclone with crypt storage, upload file to webdav server, no error.

[webdav]
type = webdav
url = http://xxxxx:5244/dav/

[crypt]
type = crypt
remote =webdav:/123pan/test/
filename_encryption = standard
directory_name_encryption = true
password = ......
password2 = ......

root@debian:~# rclone copy test10.txt crypt:/ -vv
2024/08/11 17:00:30 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test10.txt" "crypt:/" "-vv"]
2024/08/11 17:00:30 DEBUG : Creating backend with remote "test10.txt"
2024/08/11 17:00:30 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 17:00:30 DEBUG : fs cache: adding new entry for parent of "test10.txt", "/root"
2024/08/11 17:00:30 DEBUG : Creating backend with remote "crypt:/"
2024/08/11 17:00:30 DEBUG : Creating backend with remote "webdav:/123pan/test"
2024/08/11 17:00:30 DEBUG : found headers: 
2024/08/11 17:00:30 DEBUG : fs cache: renaming cache item "webdav:/123pan/test" to be canonical "webdav:123pan/test"
2024/08/11 17:00:30 DEBUG : fs cache: switching user supplied name "webdav:/123pan/test" for canonical name "webdav:123pan/test"
2024/08/11 17:00:30 DEBUG : test10.txt: Need to transfer - File not found at Destination
2024/08/11 17:00:30 DEBUG : test10.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:00:32 INFO  : test10.txt: Copied (new)
2024/08/11 17:00:32 INFO  : 
Transferred:   	         59 B / 59 B, 100%, 58 B/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         1.2s

2024/08/11 17:00:32 DEBUG : 8 go routines active
2

Looks like this remote has issue with server-side operations which are required by default chunker transactions type.

Setting --chunker-transactions to norename should fix the issue.

3

still don't work.

[webdav]
type = webdav
url = http://192.168.3.195:5244/dav/

[chunker]
type = chunker
remote = webdav:/123pan/test/
chunk_size =3800M
hash_type = sha1
name_format = *.#
transactions = norename

root@debian:~# rclone copy test15.txt chunker:/ -vv
2024/08/11 17:22:28 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test15.txt" "chunker:/" "-vv"]
2024/08/11 17:22:28 DEBUG : Creating backend with remote "test15.txt"
2024/08/11 17:22:28 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 17:22:28 DEBUG : fs cache: adding new entry for parent of "test15.txt", "/root"
2024/08/11 17:22:28 DEBUG : Creating backend with remote "chunker:/"
2024/08/11 17:22:28 DEBUG : Creating backend with remote "webdav:/123pan/test"
2024/08/11 17:22:28 DEBUG : found headers: 
2024/08/11 17:22:28 DEBUG : fs cache: renaming cache item "webdav:/123pan/test" to be canonical "webdav:123pan/test"
2024/08/11 17:22:28 DEBUG : Reset feature "ListR"
2024/08/11 17:22:28 DEBUG : test15.txt: Need to transfer - File not found at Destination
2024/08/11 17:22:28 DEBUG : test15.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:22:28 DEBUG : test15.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:22:29 ERROR : test15.txt.1_1wwyuq: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:22:29 ERROR : test15.txt.1_1wwyuq: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:22:29 ERROR : test15.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:22:29 ERROR : Attempt 1/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:22:29 DEBUG : test15.txt: Need to transfer - File not found at Destination
2024/08/11 17:22:29 DEBUG : test15.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:22:29 DEBUG : test15.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:22:31 ERROR : test15.txt.1_1wwzyj: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:22:31 ERROR : test15.txt.1_1wwzyj: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:22:31 ERROR : test15.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:22:31 ERROR : Attempt 2/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:22:31 DEBUG : test15.txt: Need to transfer - File not found at Destination
2024/08/11 17:22:31 DEBUG : test15.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:22:31 DEBUG : test15.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:22:32 ERROR : test15.txt.1_1wx179: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:22:32 ERROR : test15.txt.1_1wx179: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:22:32 ERROR : test15.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:22:32 ERROR : Attempt 3/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:22:32 INFO  : 
Transferred:   	         24 B / 24 B, 100%, 5 B/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         4.7s

2024/08/11 17:22:32 DEBUG : 8 go routines active
2024/08/11 17:22:32 Failed to copy: copy NewObject failed: object not found
4

I said norename:slight_smile:

5

I made a mistake but it's still the same

[webdav]
type = webdav
url = http://192.168.3.195:5244/dav/

[chunker]
type = chunker
remote = webdav:/123pan/test/
chunk_size =3800M
hash_type = sha1
name_format = *.#
transactions = norename

root@debian:~# rclone copy test16.txt chunker:/ -vv
2024/08/11 17:24:15 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test16.txt" "chunker:/" "-vv"]
2024/08/11 17:24:15 DEBUG : Creating backend with remote "test16.txt"
2024/08/11 17:24:15 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 17:24:15 DEBUG : fs cache: adding new entry for parent of "test16.txt", "/root"
2024/08/11 17:24:15 DEBUG : Creating backend with remote "chunker:/"
2024/08/11 17:24:15 DEBUG : Creating backend with remote "webdav:/123pan/test"
2024/08/11 17:24:15 DEBUG : found headers: 
2024/08/11 17:24:15 DEBUG : fs cache: renaming cache item "webdav:/123pan/test" to be canonical "webdav:123pan/test"
2024/08/11 17:24:15 DEBUG : Reset feature "ListR"
2024/08/11 17:24:15 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:24:15 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:24:15 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:24:16 ERROR : test16.txt.1_1wzxbb: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:24:16 ERROR : test16.txt.1_1wzxbb: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:24:16 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:24:16 ERROR : Attempt 1/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:24:16 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:24:16 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:24:16 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:24:18 ERROR : test16.txt.1_1wzyqa: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:24:18 ERROR : test16.txt.1_1wzyqa: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:24:18 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:24:18 ERROR : Attempt 2/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:24:18 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:24:18 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:24:18 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:24:19 ERROR : test16.txt.1_1x00sm: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:24:19 ERROR : test16.txt.1_1x00sm: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:24:19 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:24:19 ERROR : Attempt 3/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:24:19 INFO  : 
Transferred:   	         24 B / 24 B, 100%, 5 B/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         4.6s

2024/08/11 17:24:19 DEBUG : 8 go routines active
2024/08/11 17:24:19 Failed to copy: copy NewObject failed: object not found
6

Hmmm.

Can you try now:

rclone copy test16.txt chunker: -vv

or

rclone copy test16.txt chunker:test -vv
7

my webdav server unable to create file/folder in root, so I upload to /123pan and /123pan/test instead.

 root@debian:~# rclone copy test16.txt chunker:/123pan -vv
2024/08/11 17:42:25 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test16.txt" "chunker:/123pan" "-vv"]
2024/08/11 17:42:25 DEBUG : Creating backend with remote "test16.txt"
2024/08/11 17:42:25 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 17:42:25 DEBUG : fs cache: adding new entry for parent of "test16.txt", "/root"
2024/08/11 17:42:25 DEBUG : Creating backend with remote "chunker:/123pan"
2024/08/11 17:42:25 DEBUG : Creating backend with remote "webdav:/123pan/test/123pan"
2024/08/11 17:42:25 DEBUG : found headers: 
2024/08/11 17:42:25 DEBUG : fs cache: renaming cache item "webdav:/123pan/test/123pan" to be canonical "webdav:123pan/test/123pan"
2024/08/11 17:42:25 DEBUG : Reset feature "ListR"
2024/08/11 17:42:25 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:42:25 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:42:26 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:42:29 ERROR : test16.txt.rclone_chunk.001_1xu7ef: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:42:29 ERROR : test16.txt.rclone_chunk.001_1xu7ef: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:42:29 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:42:29 ERROR : Attempt 1/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:42:29 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:42:29 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:42:29 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:42:30 ERROR : test16.txt.rclone_chunk.001_1xubqh: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:42:31 ERROR : test16.txt.rclone_chunk.001_1xubqh: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:42:31 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:42:31 ERROR : Attempt 2/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:42:31 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:42:31 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:42:31 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:42:32 ERROR : test16.txt.rclone_chunk.001_1xud0r: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:42:32 ERROR : test16.txt.rclone_chunk.001_1xud0r: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:42:32 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:42:32 ERROR : Attempt 3/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:42:32 INFO  : 
Transferred:   	         24 B / 24 B, 100%, 3 B/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         6.9s

2024/08/11 17:42:32 DEBUG : 8 go routines active
2024/08/11 17:42:32 Failed to copy: copy NewObject failed: object not found

root@debian:~# rclone copy test16.txt chunker:/123pan/test -vv
2024/08/11 17:42:36 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test16.txt" "chunker:/123pan/test" "-vv"]
2024/08/11 17:42:36 DEBUG : Creating backend with remote "test16.txt"
2024/08/11 17:42:36 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 17:42:36 DEBUG : fs cache: adding new entry for parent of "test16.txt", "/root"
2024/08/11 17:42:36 DEBUG : Creating backend with remote "chunker:/123pan/test"
2024/08/11 17:42:36 DEBUG : Creating backend with remote "webdav:/123pan/test/123pan/test"
2024/08/11 17:42:36 DEBUG : found headers: 
2024/08/11 17:42:36 DEBUG : fs cache: renaming cache item "webdav:/123pan/test/123pan/test" to be canonical "webdav:123pan/test/123pan/test"
2024/08/11 17:42:36 DEBUG : Reset feature "ListR"
2024/08/11 17:42:36 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:42:36 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:42:36 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:42:39 ERROR : test16.txt.rclone_chunk.001_1xui11: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:42:40 ERROR : test16.txt.rclone_chunk.001_1xui11: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:42:40 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:42:40 ERROR : Attempt 1/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:42:40 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:42:40 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:42:40 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:42:41 ERROR : test16.txt.rclone_chunk.001_1xumly: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:42:41 ERROR : test16.txt.rclone_chunk.001_1xumly: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:42:41 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:42:41 ERROR : Attempt 2/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:42:41 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/11 17:42:41 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/11 17:42:41 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 17:42:42 ERROR : test16.txt.rclone_chunk.001_1xunta: Couldn't move: copy NewObject failed: object not found
2024/08/11 17:42:43 ERROR : test16.txt.rclone_chunk.001_1xunta: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/11 17:42:43 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/11 17:42:43 ERROR : Attempt 3/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/11 17:42:43 INFO  : 
Transferred:   	         24 B / 24 B, 100%, 3 B/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         6.5s

2024/08/11 17:42:43 DEBUG : 8 go routines active
2024/08/11 17:42:43 Failed to copy: copy NewObject failed: object not found

and I paste the config again:

[webdav]
type = webdav
url = http://192.168.3.195:5244/dav/

[chunker]
type = chunker
remote = webdav:/123pan/test/
hash_type = sha1
transactions = norename

8

Let's see what others can spot and suggest.

@asdffdsa ?

I am not so familiar with webdav.

9

@innova, copy a single file to the webdav server, without chunker, and post that debug log.

and to keep the logs smaller, use --retries=1

10 
root@debian:~# echo 'fewfjlkwegewg' > test18.txt
root@debian:~# rclone copy test18.txt webdav:/123pan/test/ -vv --retries=1
2024/08/11 21:17:02 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test18.txt" "webdav:/123pan/test/" "-vv" "--retries=1"]
2024/08/11 21:17:02 DEBUG : Creating backend with remote "test18.txt"
2024/08/11 21:17:02 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/11 21:17:02 DEBUG : fs cache: adding new entry for parent of "test18.txt", "/root"
2024/08/11 21:17:02 DEBUG : Creating backend with remote "webdav:/123pan/test/"
2024/08/11 21:17:02 DEBUG : found headers: 
2024/08/11 21:17:02 DEBUG : fs cache: renaming cache item "webdav:/123pan/test/" to be canonical "webdav:123pan/test"
2024/08/11 21:17:02 DEBUG : test18.txt: Need to transfer - File not found at Destination
2024/08/11 21:17:02 DEBUG : test18.txt: Update will use the normal upload strategy (no chunks)
2024/08/11 21:17:02 INFO  : test18.txt: Copied (new)
2024/08/11 21:17:02 INFO  : 
Transferred:   	         14 B / 14 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.4s

2024/08/11 21:17:02 DEBUG : 9 go routines active
11

please post
rclone backend features webdav:

12 
root@debian:~#  rclone backend features webdav:
{
	"Name": "webdav",
	"Root": "",
	"String": "webdav root ''",
	"Precision": 3153600000000000000,
	"Hashes": [],
	"Features": {
		"About": true,
		"BucketBased": false,
		"BucketBasedRootOK": false,
		"CanHaveEmptyDirectories": true,
		"CaseInsensitive": false,
		"ChangeNotify": false,
		"ChunkWriterDoesntSeek": false,
		"CleanUp": false,
		"Command": false,
		"Copy": true,
		"DirCacheFlush": false,
		"DirModTimeUpdatesOnWrite": false,
		"DirMove": true,
		"DirSetModTime": false,
		"Disconnect": false,
		"DuplicateFiles": false,
		"FilterAware": false,
		"GetTier": false,
		"IsLocal": false,
		"ListR": false,
		"MergeDirs": false,
		"MkdirMetadata": false,
		"Move": true,
		"NoMultiThreading": false,
		"OpenChunkWriter": false,
		"OpenWriterAt": false,
		"Overlay": false,
		"PartialUploads": false,
		"PublicLink": false,
		"Purge": true,
		"PutStream": false,
		"PutUnchecked": false,
		"ReadDirMetadata": false,
		"ReadMetadata": false,
		"ReadMimeType": false,
		"ServerSideAcrossConfigs": false,
		"SetTier": false,
		"SetWrapper": false,
		"Shutdown": false,
		"SlowHash": false,
		"SlowModTime": false,
		"UnWrap": false,
		"UserDirMetadata": false,
		"UserInfo": false,
		"UserMetadata": false,
		"WrapFs": false,
		"WriteDirMetadata": false,
		"WriteDirSetModTime": false,
		"WriteMetadata": false,
		"WriteMimeType": false
	},
	"MetadataInfo": null
}
13

i have never used chunker but basic webdav functionality seems to work

maybe an issue with server-side moves.
tho that seems to be supported, based on "Move": true

1 Like
14

I tried to connect another webdav server, it works well.

but if I have to use this webdav server, is there any way to fix it? Is it possible to use other methods instead of the "move" method?

15

seems to be an issue with rclone+alist webdav server
https://forum.rclone.org/t/alist-error-or-rclone-error/41484

16

I have read this topic, the issue is similar, but I can upload to webdav without chunker.

17

not sure that is the issue.
i think you can test that, on webdav:, by using --backup-dir which requires server-side move/copy

might try --dump=headers for a deeper look.

18

This is your original error:

and this is with transactions = norename:

Which sort of let me believe that it is not using transactions = norename as in such case there are no temporary chunks to rename...

Are you sure you are using the right config file?

Let's try to be explicit and set it in command line - try:

rclone copy test16.txt chunker: -vv --chunker-transactions=norename
19 
root@debian:~# rclone config redacted
[chunker]
type = chunker
remote = webdav:/123pan/test/
hash_type = sha1
transactions = norename

[webdav]
type = webdav
url = http://192.168.3.21:5244/dav/
user = XXX
pass = XXX
### Double check the config for sensitive info before posting publicly
root@debian:~# 
root@debian:~# ls -lh test16.txt
-rw-r--r-- 1 root root 8 Aug 11 17:24 test16.txt
root@debian:~# 
root@debian:~# rclone ls chunker:
       14 test18.txt
root@debian:~# 
root@debian:~# rclone copy test16.txt chunker:  -vv --chunker-transactions=norename
2024/08/12 11:08:56 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test16.txt" "chunker:" "-vv" "--chunker-transactions=norename"]
2024/08/12 11:08:56 DEBUG : Creating backend with remote "test16.txt"
2024/08/12 11:08:56 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/12 11:08:56 DEBUG : fs cache: adding new entry for parent of "test16.txt", "/root"
2024/08/12 11:08:56 DEBUG : Creating backend with remote "chunker:"
2024/08/12 11:08:56 DEBUG : chunker: detected overridden config - adding "{BpqkF}" suffix to name
2024/08/12 11:08:56 DEBUG : Creating backend with remote "webdav:/123pan/test/"
2024/08/12 11:08:56 DEBUG : found headers: 
2024/08/12 11:08:56 DEBUG : fs cache: renaming cache item "webdav:/123pan/test/" to be canonical "webdav:123pan/test"
2024/08/12 11:08:56 DEBUG : Reset feature "ListR"
2024/08/12 11:08:56 DEBUG : fs cache: renaming cache item "chunker:" to be canonical "chunker{BpqkF}:"
2024/08/12 11:08:56 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/12 11:08:56 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/12 11:08:56 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/12 11:08:57 ERROR : test16.txt.rclone_chunk.001_3aae29: Couldn't move: copy NewObject failed: object not found
2024/08/12 11:08:57 ERROR : test16.txt.rclone_chunk.001_3aae29: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/12 11:08:57 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/12 11:08:57 ERROR : Attempt 1/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/12 11:08:57 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/12 11:08:57 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/12 11:08:57 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/12 11:08:58 ERROR : test16.txt.rclone_chunk.001_3aafys: Couldn't move: copy NewObject failed: object not found
2024/08/12 11:08:59 ERROR : test16.txt.rclone_chunk.001_3aafys: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/12 11:08:59 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/12 11:08:59 ERROR : Attempt 2/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/12 11:08:59 DEBUG : test16.txt: Need to transfer - File not found at Destination
2024/08/12 11:08:59 DEBUG : test16.txt: skip slow SHA1 on source file, hashing in-transit
2024/08/12 11:08:59 DEBUG : test16.txt: Update will use the normal upload strategy (no chunks)
2024/08/12 11:09:00 ERROR : test16.txt.rclone_chunk.001_3aahpk: Couldn't move: copy NewObject failed: object not found
2024/08/12 11:09:00 ERROR : test16.txt.rclone_chunk.001_3aahpk: Failed to remove temporary chunk: Not Found: 404 Not Found
2024/08/12 11:09:00 ERROR : test16.txt: Failed to copy: copy NewObject failed: object not found
2024/08/12 11:09:00 ERROR : Attempt 3/3 failed with 1 errors and: copy NewObject failed: object not found
2024/08/12 11:09:00 INFO  : 
Transferred:   	         24 B / 24 B, 100%, 5 B/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:                 3 / 3, 100%
Elapsed time:         4.2s

2024/08/12 11:09:00 DEBUG : 8 go routines active
2024/08/12 11:09:00 Failed to copy: copy NewObject failed: object not found
20

on webdav:, by using --backup-dir

root@debian:~# rclone ls webdav:/123pan/test
       14 test18.txt
root@debian:~# 
root@debian:~# rclone ls webdav:/123pan/test/backup
root@debian:~# 
root@debian:~# rclone copy test18.txt webdav:/123pan/test/  -vv --chunker-transactions=norename --backup-dir=webdav:/123pan/test/backup/
2024/08/12 11:27:47 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "test18.txt" "webdav:/123pan/test/" "-vv" "--chunker-transactions=norename" "--backup-dir=webdav:/123pan/test/backup/"]
2024/08/12 11:27:47 DEBUG : Creating backend with remote "test18.txt"
2024/08/12 11:27:47 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/08/12 11:27:47 DEBUG : fs cache: adding new entry for parent of "test18.txt", "/root"
2024/08/12 11:27:47 DEBUG : Creating backend with remote "webdav:/123pan/test/"
2024/08/12 11:27:47 DEBUG : found headers: 
2024/08/12 11:27:47 DEBUG : fs cache: renaming cache item "webdav:/123pan/test/" to be canonical "webdav:123pan/test"
2024/08/12 11:27:47 DEBUG : Creating backend with remote "webdav:/123pan/test/backup/"
2024/08/12 11:27:47 DEBUG : found headers: 
2024/08/12 11:27:47 DEBUG : fs cache: renaming cache item "webdav:/123pan/test/backup/" to be canonical "webdav:123pan/test/backup"
2024/08/12 11:27:47 DEBUG : test18.txt: Sizes differ (src 19 vs dst 14)
2024/08/12 11:27:48 INFO  : test18.txt: Moved (server-side)
2024/08/12 11:27:48 DEBUG : test18.txt: Update will use the normal upload strategy (no chunks)
2024/08/12 11:27:49 INFO  : test18.txt: Copied (new)
2024/08/12 11:27:49 INFO  : 
Transferred:   	         19 B / 19 B, 100%, 18 B/s, ETA 0s
Checks:                 1 / 1, 100%
Renamed:                1
Transferred:            1 / 1, 100%
Server Side Moves:      1 @ 14 B
Elapsed time:         1.6s

2024/08/12 11:27:49 DEBUG : 8 go routines active