Hmm, duplicates appearing on pCloud may be a bug in pCloud - as far as I know it isn't supposed to have duplicates.
Did you ask them about it?
Not yet; I tried another client (which doesn't say anything about duplicates!?), and then performed a >>% curl -I https://webdav.pcloud.com -u email@somewhere.com
Enter host password for user 'email@somewhere.com':
HTTP/1.1 200 OK
Date: Tue, 03 Jan 2023 06:39:40 GMT
Server: Apache/2.4.10 (Debian)
Content-Type: text/html;charset=UTF-8<<
So apparently they're just using the Apache WebDav modules (!?), which, given that my server is running a similar Apache 2.4.38, I decided to install... seems to be working fine on my server, but I have only tried a single file, so far...
Well, when I switched over to using a 'pCloud API' source, I had been able to 'rclone copy' those files to a local directory without visible errors (and dedupe w/hash started working); however, when attempting to 'rclone copy' the zlib files to my new Apache WebDav mount, I get a whole series of multx errors with 12 files:
"...404 Not Found 2023/01/03 13:03:01 ERROR : contrib/dotzlib/DotZLib/AssemblyInfo.cs: Failed to copy: Update mkParentDir failed: 400 Bad Request
Bad Request
Your browser sent a request that this server could not understand.
... 2023/01/03 13:03:27 ERROR : Attempt 3/3 failed with 12 errors and: ..."
However, it does appear to have copied 384 of the files across ok...
I'd say that it appears to me that rclone webdav support could use some work! FWIW, I think maybe you should even have specialized 'vendor' types for apache variants, as it looks like mod_dav has been one of their favorite things to tweak : Apache 2.4 Change Log
FYI, my apache webdav sites-enabled ssl config (FWIW, apparently working fine with 3 major browsers & 'CyberDuck'):
DavLockDB /var/www/html/DavLockDir/DavLock
Alias /webdav /var/www/html/webdav
<Directory /var/www/html/webdav>
DAV On
# AuthType Digest -- not working with rclone yet
AuthType Basic
AuthName "webdav"
# AuthUserFile /etc/apache2/.htpasswd -- not working with rclone yet
AuthUserFile /etc/apache2/.htpasswd_basic
Require valid-user
...
</Directory>
pCloud SDK files in question available here:
(FWIW, I did try the "--transfers 1 --checkers 1 --check-first" that I saw that Craig had suggested elsewhere on the 'Update mkParentDir failed' errors elsewhere, but still happens)
This is possibly rclone trying to create an alread existing directory and getting an error back. The error should be one of these ones (from the webdav backend code)
if apiErr.StatusCode == http.StatusMethodNotAllowed || apiErr.StatusCode == http.StatusNotAcceptable || apiErr.StatusCode == http.StatusLocked {
return nil
}
But maybe apache is returning yet another error code!
If you capture what happens with -vv --dump bodies we can probably work it out.
Note webdav is a very badly standardised protocol and the backend is full of hacks and bodges for different webdav servers 
Ah, well I looked at your system's recommendation of 'pastebin', but it only accepts up to 512k & the resultant log file seems to be ~several times more than that.
I have 'rclone copy'd from WebDav source to WebDav destination, in the hopes that you may spot sufficient errors from each before it failed to complete.
Here, I've put it up on one of my servers for the moment (not sure how long it'll remain there):
https://arma.jeddo.in/Secure/rclone_webdav_errlog1.txt
Here is an example of a failed request and response
2023/01/04 14:09:09 DEBUG : HTTP REQUEST (req 0xc000770700)
2023/01/04 14:09:09 DEBUG : PROPFIND /webdav/zlib-1.2.8/contrib/dotzlib/DotZLib/ HTTP/1.1
Host: SomeDomain.com
User-Agent: rclone/
Authorization: XXXX
Depth: 0
Referer: https://SomeDomain.com/webdav/
Accept-Encoding: gzip
and the response
2023/01/04 14:09:11 DEBUG : HTTP RESPONSE (req 0xc000770700)
2023/01/04 14:09:11 DEBUG : HTTP/1.1 400 Bad Request
Connection: close
Content-Length: 303
Content-Type: text/html; charset=iso-8859-1
Date: Wed, 04 Jan 2023 21:09:09 GMT
Server: Apache/2.4.38 (Debian)
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache/2.4.38 (Debian) Server at SomeDomain.com Port 443</address>
</body></html>
2023/01/04 14:09:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/01/04 14:09:11 ERROR : contrib/dotzlib/DotZLib/ChecksumImpl.cs: Failed to copy: Update mkParentDir failed: ... HTML page repeated
The request looks perfectly valid to me and shouldn't have had a 400 response. So this looks like a bug in the server to me.
It might be that rclone's concurrency is confusing the server - try --checkers 1 --transfers 1 --check-first to reduce the concurrency to the minimum and see what happens.
Hmm; for some reason, it appears to be failing to create the DotZLib directory in general...
I tried erasing the main zlib dir & starting over (88M log!), and then copying over that (3.3M):
https://arma.jeddo.in/Secure/rclone_webdav_errlog2.txt
I thought it was noteworthy that it tried to create the main zlib directory more than once!?
https://arma.jeddo.in/Secure/rclone_webdav_errlog3.txt
FWIW, You can see in errlog3 the first "MKCOL /webdav/zlib-1.2.8/contrib/dotzlib/DotZLib/" fails with a "400 Bad Request", unlike the original MKCOL for the /webdav/zlib-1.2.8/ directory itself in the errlog2.
Ah! well, I appear to have found ways which may address some of the issues found subsequent to the original report:
Failure to create the 'DotZLib' appears to be associated with this 'stackoverflow' report:
https://stackoverflow.com/questions/20695683/apache-webdav-unable-to-create-test-folder-when-file-test-txt-exists
However, if I try their recommendation of setting 'Options None', rclone fails (as do other clients) when attempting to get a directory listing; so I found out that just making sure that the 'MultiViews' (pattern matching?) option is turned off appears to allow the directory creation, with ~ extant files of similar names, in this case. {I did appear to see many duplicate 'MKCOL's when subsequently performing the 'rclone copy' in the output log however (!?)}.
The issue of "rclone copy pCloud_WebDav:/Shared\ Folder/pSDKBuild/Library/libs/" failing due to "Duplicate object found in source" appears to have stopped happening [or maybe network traffic isn't as bad now, or something?] after I ran the 'dedupe' (which only worked using 'pCloud API' remote w/hash), though no duplicates had ever been shown when 'rclone ls/f' was run on the directories in question.
As for the original report of errors when moving from a WebDav pCloud crypt remote into a subdirectory of it, I haven't retried it yet (I ended up going for a rsync backup to a laptop client, compressing to a tgz & uploading instead). I currently suspect that a 'rsync dedupe' (using 'pCloud API' remote w/hash) might somehow tweak something invisible there too.
Yes this is a known failing of the webdav backend. The webdav server should just report that the directory already exists so all we've wasted is a transaction.
This could be exacerbating the problem though - i'm not sure.
Still showing many significant errors with rclone & pcloud's webdav when I just happened to do a move to a remote which I hadn't switched over to their direct API yet:
rclone -v move ./gmime_3.2.11 BackupCrypt:/tmpdev_archived
2023/01/20 17:31:33 NOTICE: README: Can't follow symlink without -L/--copy-links
2023/01/20 17:31:34 ERROR : .gitignore: corrupted on transfer: sizes differ 329 vs 0
2023/01/20 17:31:34 INFO : .gitignore: Removing failed copy
2023/01/20 17:31:34 ERROR : .travis.yml: corrupted on transfer: sizes differ 669 vs 0
2023/01/20 17:31:34 INFO : .travis.yml: Removing failed copy
2023/01/20 17:31:34 ERROR : COPYING: corrupted on transfer: sizes differ 26436 vs 0
2023/01/20 17:31:34 INFO : COPYING: Removing failed copy
2023/01/20 17:31:34 ERROR : .gitignore: Not deleting source as copy failed: corrupted on transfer: sizes differ 329 vs 0
2023/01/20 17:31:34 ERROR : .travis.yml: Not deleting source as copy failed: corrupted on transfer: sizes differ 669 vs 0
2023/01/20 17:31:34 INFO : AUTHORS: Copied (new)
2023/01/20 17:31:34 INFO : AUTHORS: Deleted
2023/01/20 17:31:34 ERROR : COPYING: Not deleting source as copy failed: corrupted on transfer: sizes differ 26436 vs 0
2023/01/20 17:31:34 INFO : ChangeLog: Copied (new)
2023/01/20 17:31:34 INFO : ChangeLog: Deleted
2023/01/20 17:31:34 INFO : LICENSE: Copied (new)
2023/01/20 17:31:34 INFO : LICENSE: Deleted
2023/01/20 17:31:35 ERROR : HACKING: corrupted on transfer: sizes differ 208 vs 0
2023/01/20 17:31:35 INFO : HACKING: Removing failed copy
2023/01/20 17:31:35 INFO : Makefile.am: Copied (new)
2023/01/20 17:31:35 INFO : Makefile.am: Deleted
2023/01/20 17:31:35 ERROR : HACKING: Not deleting source as copy failed: corrupted on transfer: sizes differ 208 vs 0
2023/01/20 17:31:35 INFO : PORTING: Copied (new)
2023/01/20 17:31:35 INFO : PORTING: Deleted
2023/01/20 17:31:35 INFO : RFCs.md: Copied (new)
2023/01/20 17:31:35 INFO : RFCs.md: Deleted
2023/01/20 17:31:35 INFO : TODO: Copied (new)
2023/01/20 17:31:35 INFO : TODO: Deleted
2023/01/20 17:31:35 ERROR : README.md: corrupted on transfer: sizes differ 32699 vs 0
2023/01/20 17:31:35 INFO : README.md: Removing failed copy
2023/01/20 17:31:35 ERROR : README.md: Not deleting source as copy failed: corrupted on transfer: sizes differ 32699 vs 0
2023/01/20 17:31:35 INFO : config.rpath: Copied (new)
2023/01/20 17:31:35 INFO : config.rpath: Deleted
2023/01/20 17:31:35 INFO : autogen.sh: Copied (new)
2023/01/20 17:31:35 INFO : autogen.sh: Deleted
2023/01/20 17:31:35 INFO : configure.ac: Copied (new)
2023/01/20 17:31:35 INFO : configure.ac: Deleted
2023/01/20 17:31:36 ERROR : fedora.setup: corrupted on transfer: sizes differ 345 vs 0
2023/01/20 17:31:36 INFO : fedora.setup: Removing failed copy
2023/01/20 17:31:36 INFO : NEWS: Copied (new)
2023/01/20 17:31:36 INFO : NEWS: Deleted
2023/01/20 17:31:36 ERROR : fedora.setup: Not deleting source as copy failed: corrupted on transfer: sizes differ 345 vs 0
2023/01/20 17:31:36 INFO : gmime.doap: Copied (new)
2023/01/20 17:31:36 INFO : gmime.doap: Deleted
2023/01/20 17:31:36 INFO : gmime.spec.in: Copied (new)
2023/01/20 17:31:36 INFO : gmime.spec.in: Deleted
2023/01/20 17:31:36 ERROR : iconv-detect.c: corrupted on transfer: sizes differ 7022 vs 0
2023/01/20 17:31:36 INFO : iconv-detect.c: Removing failed copy
2023/01/20 17:31:36 ERROR : iconv-detect.c: Not deleting source as copy failed: corrupted on transfer: sizes differ 7022 vs 0
...
I think y'all might want to get one of their IMO relatively inexpensive ('lifetime'!) accounts & do some testing, as it would seem that the apache webdav that they're using should likely be more supportable!...
That is a bit sad...
I have a free account - do they support webdav on that?
Apparently no webdav available on their free accounts. FWIW, I got their 'lifetime' accounts on one of their "80%" off sales, which I've seen go by multiple times this year.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.