Can't manage folders using Bitrix webdav under Windows

moccij · March 25, 2021, 1:30pm

What is the problem you are having with rclone?

I have mounted Bitrix24.Drive as a Windows drive with rclone using the webdav backend. Renaming and moving folders does not work. However, I can create and delete folders.

The exact error is:

2021/03/25 14:19:24 ERROR : PRODUZIONE/PRODOTTI/PHOTON_LOOP/Nuova cartella: Dir.Rename error: DirMove MOVE call failed: 404 Not Found
2021/03/25 14:19:24 ERROR : IO error: DirMove MOVE call failed: 404 Not Found

What is your rclone version (output from `rclone version`)

rclone v1.54.1
- os/arch: windows/amd64
- go version: go1.15.8

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Bitrix24 (Bitrix24.Drive, webdav)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

mount Bitrix: o: --vfs-cache-mode full --file-perms=0777 --no-console --log-file="C:\users\jacop\documents\rclone\rclone.txt"

The rclone config contents with secrets removed.

[Bitrix]
type = webdav
vendor = Bitrix24
pass = ***
url = ***
user = ***
bearer_token =

A log from the command with the `-vv` flag

Broken down to actually useful parts.

https://pastebin.com/Ku3KMR7X
pass: bitrix2021

asdffdsa · March 25, 2021, 1:44pm

hello and welcome to the forum

using the command line, not rclone mount when you you try to move a folder, what happens?

there are other posts about bitrix, for example,
Bitrix24.de remote support?

moccij · March 25, 2021, 2:37pm

If I try rclone move on a folder, I get:

2021/03/25 15:29:27 DEBUG : rclone: Version "v1.54.1" starting with parameters ["C:\\Users\\jacop\\Documents\\rclone\\rclone.exe" "move" "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\Nuova Cartella" "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test" "-vv"]
2021/03/25 15:29:27 DEBUG : Using config file from "C:\\Users\\jacop\\.config\\rclone\\rclone.conf"
2021/03/25 15:29:27 DEBUG : Creating backend with remote "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\Nuova Cartella"
2021/03/25 15:29:27 DEBUG : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/Nuova Cartella': Unknown vendor "Bitrix24"
2021/03/25 15:29:27 DEBUG : fs cache: renaming cache item "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\Nuova Cartella" to be canonical "Bitrix:PRODUZIONE/PRODOTTI/PHOTON_LOOP/Nuova Cartella"
2021/03/25 15:29:27 DEBUG : Creating backend with remote "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test"
2021/03/25 15:29:27 DEBUG : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test': Unknown vendor "Bitrix24"
2021/03/25 15:29:27 DEBUG : fs cache: renaming cache item "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test" to be canonical "Bitrix:PRODUZIONE/PRODOTTI/PHOTON_LOOP/test"
2021/03/25 15:29:27 DEBUG : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test': Using server-side directory move
2021/03/25 15:29:27 ERROR : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test': Server side directory move failed: DirMove MOVE call failed: 404 Not Found
2021/03/25 15:29:27 ERROR : Attempt 1/3 failed with 1 errors and: DirMove MOVE call failed: 404 Not Found
2021/03/25 15:29:27 DEBUG : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test': Using server-side directory move
2021/03/25 15:29:28 ERROR : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test': Server side directory move failed: DirMove MOVE call failed: 404 Not Found
2021/03/25 15:29:28 ERROR : Attempt 2/3 failed with 1 errors and: DirMove MOVE call failed: 404 Not Found
2021/03/25 15:29:28 DEBUG : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test': Using server-side directory move
2021/03/25 15:29:28 ERROR : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test': Server side directory move failed: DirMove MOVE call failed: 404 Not Found
2021/03/25 15:29:28 ERROR : Attempt 3/3 failed with 1 errors and: DirMove MOVE call failed: 404 Not Found
2021/03/25 15:29:28 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         1.5s

2021/03/25 15:29:28 DEBUG : 4 go routines active
2021/03/25 15:29:28 Failed to move: DirMove MOVE call failed: 404 Not Found

However, rclone move on files is fine:

2021/03/25 15:28:29 DEBUG : rclone: Version "v1.54.1" starting with parameters ["C:\\Users\\jacop\\Documents\\rclone\\rclone.exe" "move" "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test.txt" "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test2.txt" "-vv"]
2021/03/25 15:28:29 DEBUG : Using config file from "C:\\Users\\jacop\\.config\\rclone\\rclone.conf"
2021/03/25 15:28:29 DEBUG : Creating backend with remote "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test.txt"
2021/03/25 15:28:29 DEBUG : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test.txt': Unknown vendor "Bitrix24"
2021/03/25 15:28:30 DEBUG : fs cache: adding new entry for parent of "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test.txt", "Bitrix:PRODUZIONE/PRODOTTI/PHOTON_LOOP"
2021/03/25 15:28:30 DEBUG : Creating backend with remote "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test2.txt"
2021/03/25 15:28:30 DEBUG : webdav root 'PRODUZIONE/PRODOTTI/PHOTON_LOOP/test2.txt': Unknown vendor "Bitrix24"
2021/03/25 15:28:30 DEBUG : fs cache: renaming cache item "Bitrix:PRODUZIONE\\PRODOTTI\\PHOTON_LOOP\\test2.txt" to be canonical "Bitrix:PRODUZIONE/PRODOTTI/PHOTON_LOOP/test2.txt"
2021/03/25 15:28:30 DEBUG : test.txt: Need to transfer - File not found at Destination
2021/03/25 15:28:30 INFO  : test.txt: Moved (server-side)
2021/03/25 15:28:30 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 1 / 1, 100%
Renamed:                1
Elapsed time:         0.9s

2021/03/25 15:28:30 DEBUG : 4 go routines active

I am aware of the other Bitrix24 post in the forum, but I did not have problems in reaching it from the webdav interface, and also the post was closed.

The most similar post I encountered is this:
unable-to-rename-and-move-directories-via-webdav (can't include links for some reason)

However, in that post the error is different: 409, conflict.

ncw · March 25, 2021, 4:44pm

I can replicate that (I have a test bitrix account)

The problem is this HTTP transaction

2021/03/25 16:39:09 DEBUG : HTTP REQUEST (req 0xc00064c800)
2021/03/25 16:39:09 DEBUG : MOVE /company/personal/user/1/disk/path/ncw-test/ HTTP/1.1
Host: myhost.bitrix24.de
User-Agent: rclone/v1.55.0-beta.5341.101e23d89.fix-multipart-context
Authorization: XXXX
Destination: https://myhost.bitrix24.de/company/personal/user/1/disk/path/ncw-test2/
Overwrite: F
Referer: https://myhost.bitrix24.de/company/personal/user/1/disk/path/
Accept-Encoding: gzip

2021/03/25 16:39:09 DEBUG : HTTP RESPONSE (req 0xc00064c800)
2021/03/25 16:39:09 DEBUG : HTTP/2.0 404 Not Found
Cache-Control: no-store, no-cache, must-revalidate
Content-Type: text/html; charset=UTF-8
Date: Thu, 25 Mar 2021 16:39:09 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
P3p: policyref="/bitrix/p3p.xml", CP="NON DSP COR CUR ADM DEV PSA PSD OUR UNR BUS UNI COM NAV INT DEM STA"
Pragma: no-cache
Server: nginx/1.16.1
Set-Cookie: PHPSESSID=AH5hus4cTKTq8WWcP8UcY9o44Mco22MJ; path=/; secure; HttpOnly
Set-Cookie: BITRIX_SM_PK=%2F1%2Fe536feb20d1c11a223cab89a5d07dedf; path=/; HttpOnly
Set-Cookie: BITRIX_SM_PK=%2F1%2Fe536feb20d1c11a223cab89a5d07dedf; path=/; HttpOnly
X-Dav-Powered-By: Bitrix Site Manager
X-Powered-Cms: Bitrix Site Manager (bc2cad9153cb418bb2dfd5602c3c3754)
X-Webdav-Status: 404 Not Found

I don't know what causes that - I suspect a bitrix bug.

You can work around with --disable DirMove which stops rclone trying to do server side directory renames. It will rename each file individually. I'm not sure how well that will work with a mount though.

moccij · March 25, 2021, 5:19pm

Thank you for the help.

I can confirm that --disable DirMove is a doable workaround when the folder does not have many files inside. It can become potentially very slow, unless the simultaneous transfer number is cranked to several tens at once. At least now it can do it!

ncw · March 28, 2021, 5:26pm

Great - glad the workaround works.

Does webdav rename folders work in other things with bitrix?

moccij · March 29, 2021, 7:19am

The native Windows webdav implementation can rename Bitrix folders without problems.

ncw · March 29, 2021, 6:36pm

I managed to figure out what bitrix doesn't like - it is the trailing / in the Destination: header.

Can you give this a go?

v1.55.0-beta.5347.9ea209a9a.fix-bitrix-dirmove on branch fix-bitrix-dirmove (uploaded in 15-30 mins)

This is non-RFC compliant - the RFC says that collections (directories) must always be referenced with a URL with a trailing /

Do you think you could report this as a bug to bitrix?

I could put in a quirk for bitrix as a hack if it works for you too.

moccij · March 30, 2021, 7:05am

Tried it, but now can't access the drive itself.

Startup:

2021/03/30 08:58:01 DEBUG : Using config file from "C:\\Users\\jacop\\.config\\rclone\\rclone.conf"
2021/03/30 08:58:01 DEBUG : rclone: Version "v1.55.0-beta.5347.9ea209a9a.fix-bitrix-dirmove" starting with parameters ["C:\\Users\\jacop\\Documents\\rclone\\rclone.exe" "mount" "Bitrix:" "o:" "--vfs-cache-mode" "full" "--file-perms=0777" "--rc" "--log-level=DEBUG" "--log-file=./rclone.txt"]
2021/03/30 08:58:01 NOTICE: Serving remote control on http://localhost:5572/
2021/03/30 08:58:01 DEBUG : Creating backend with remote "Bitrix:"
2021/03/30 08:58:01 DEBUG : webdav root '': Unknown vendor "Bitrix24"
2021/03/30 08:58:01 INFO  : webdav root '': poll-interval is not supported by this remote
2021/03/30 08:58:01 DEBUG : vfs cache: root is "\\\\?\\C:\\Users\\jacop\\AppData\\Local\\rclone\\vfs\\Bitrix"
2021/03/30 08:58:01 DEBUG : vfs cache: metadata root is "\\\\?\\C:\\Users\\jacop\\AppData\\Local\\rclone\\vfs\\Bitrix"
2021/03/30 08:58:01 DEBUG : Creating backend with remote "\\\\?\\C:\\Users\\jacop\\AppData\\Local\\rclone\\vfs\\Bitrix"
2021/03/30 08:58:01 DEBUG : fs cache: renaming cache item "\\\\?\\C:\\Users\\jacop\\AppData\\Local\\rclone\\vfs\\Bitrix" to be canonical "//?/C:/Users/jacop/AppData/Local/rclone/vfs/Bitrix"
2021/03/30 08:58:01 DEBUG : fs cache: switching user supplied name "\\\\?\\C:\\Users\\jacop\\AppData\\Local\\rclone\\vfs\\Bitrix" for canonical name "//?/C:/Users/jacop/AppData/Local/rclone/vfs/Bitrix"
2021/03/30 08:58:01 DEBUG : Network mode mounting is disabled
2021/03/30 08:58:01 DEBUG : Mounting on "o:" ("Bitrix")
2021/03/30 08:58:01 DEBUG : webdav root '': Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=Bitrix"]
2021/03/30 08:58:01 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2021/03/30 08:58:01 DEBUG : webdav root '': Init: 
2021/03/30 08:58:01 DEBUG : webdav root '': >Init: 
2021/03/30 08:58:01 DEBUG : /: Statfs: 
2021/03/30 08:58:02 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/03/30 08:58:02 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/03/30 08:58:02 DEBUG : /: >Getattr: errc=0
2021/03/30 08:58:02 DEBUG : /: Readlink: 
2021/03/30 08:58:02 DEBUG : /: >Readlink: linkPath="", errc=-40
The service rclone has been started.

Maybe it is unrelated, but I guess that some </a> is now missing its slash:

2021/03/30 08:58:02 DEBUG : /Images/Camera: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/03/30 08:58:02 ERROR : /: Dir.Stat error: couldn't list files: XML syntax error on line 20: element <a> closed by </td>
2021/03/30 08:58:02 ERROR : IO error: couldn't list files: XML syntax error on line 20: element <a> closed by </td>


2021/03/30 08:58:06 DEBUG : /: Opendir: 
2021/03/30 08:58:06 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/03/30 08:58:06 ERROR : /: Dir.Stat error: couldn't list files: XML syntax error on line 20: element <a> closed by </td>
2021/03/30 08:58:06 ERROR : IO error: couldn't list files: XML syntax error on line 20: element <a> closed by </td>

For the bug report, is it enough to copy the first post of this topic?

ncw · March 30, 2021, 3:47pm

looks like that attempt failed!

That's fine. Link this forum post from the issue and link the issue from this forum post - thanks!

system · May 30, 2021, 11:48am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.