Getting 404 error with pcloud (mount or copy)

Help and Support
1

What is the problem you are having with rclone?

Hi, I'm having a problem with missing files on a pcloud remote.

I'm trying to backup my pcloud files locally, and getting a 404 for a random selection of files.

If I use rclone mount and try to cat the file, I see a 404 error in the rclone log, and I get a read error trying to cat the file.

Run the command 'rclone version' and share the full output of the command.

Made sure I'm at the latest release;

# rclone version
rclone v1.61.1
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.15.0-57-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

pcloud

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -Pvv copy pcloud:Training/ ./

The rclone config contents with secrets removed.

[pcloud]
type = pcloud
client_id =
client_secret =
token = {"access_token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","token_type":"bearer","expiry":"0001-01-01T00:00:00Z"}

A log from the command with the -vv flag

I trimmed a few thousand lines of "Unchanged skipping"...

# rclone -Pvv copy pcloud:Training/ ./
2023/02/04 22:14:26 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "-Pvv" "copy" "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/" "./"]
2023/02/04 22:14:26 DEBUG : Creating backend with remote "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/"
2023/02/04 22:14:26 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/02/04 22:14:28 DEBUG : fs cache: renaming cache item "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/" to be canonical "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language"
2023/02/04 22:14:28 DEBUG : Creating backend with remote "./"
2023/02/04 22:14:28 DEBUG : fs cache: renaming cache item "./" to be canonical "/export/backup/pcloud/Training/golang/Learn How To Code Google's Go (golang) Programming Language"
2023-02-04 22:14:34 DEBUG : Couldn't parse Date: from server vc549.pcloud.com: "Sat, 04 Feb 2023 11:14:33 +0000": parsing time "Sat, 04 Feb 2023 11:14:33 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 04 Feb 2023 11:14:33 +0000" as " "
2023-02-04 22:14:34 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2023-02-04 22:14:34 ERROR : 37. Control Flow/6. Rune.srt: Failed to copy: failed to open source object: pcloud error: 404 Not Found (404)
2023-02-04 22:14:34 DEBUG : Local file system at /export/backup/pcloud/Training/golang/Learn How To Code Google's Go (golang) Programming Language: Waiting for checks to finish
2023-02-04 22:14:34 DEBUG : Local file system at /export/backup/pcloud/Training/golang/Learn How To Code Google's Go (golang) Programming Language: Waiting for transfers to finish
2023-02-04 22:14:34 ERROR : Attempt 1/3 failed with 1 errors and: failed to open source object: pcloud error: 404 Not Found (404)
2023-02-04 22:14:40 DEBUG : Couldn't parse Date: from server vc549.pcloud.com: "Sat, 04 Feb 2023 11:14:40 +0000": parsing time "Sat, 04 Feb 2023 11:14:40 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 04 Feb 2023 11:14:40 +0000" as " "
2023-02-04 22:14:40 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2023-02-04 22:14:40 ERROR : 37. Control Flow/6. Rune.srt: Failed to copy: failed to open source object: pcloud error: 404 Not Found (404)
2023-02-04 22:14:41 DEBUG : Local file system at /export/backup/pcloud/Training/golang/Learn How To Code Google's Go (golang) Programming Language: Waiting for checks to finish
2023-02-04 22:14:41 DEBUG : Local file system at /export/backup/pcloud/Training/golang/Learn How To Code Google's Go (golang) Programming Language: Waiting for transfers to finish
2023-02-04 22:14:41 ERROR : Attempt 2/3 failed with 1 errors and: failed to open source object: pcloud error: 404 Not Found (404)
2023-02-04 22:14:46 DEBUG : Couldn't parse Date: from server vc549.pcloud.com: "Sat, 04 Feb 2023 11:14:46 +0000": parsing time "Sat, 04 Feb 2023 11:14:46 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 04 Feb 2023 11:14:46 +0000" as " "
2023-02-04 22:14:46 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2023-02-04 22:14:46 ERROR : 37. Control Flow/6. Rune.srt: Failed to copy: failed to open source object: pcloud error: 404 Not Found (404)
2023-02-04 22:14:47 DEBUG : Local file system at /export/backup/pcloud/Training/golang/Learn How To Code Google's Go (golang) Programming Language: Waiting for checks to finish
2023-02-04 22:14:47 DEBUG : Local file system at /export/backup/pcloud/Training/golang/Learn How To Code Google's Go (golang) Programming Language: Waiting for transfers to finish
2023-02-04 22:14:47 ERROR : Attempt 3/3 failed with 1 errors and: failed to open source object: pcloud error: 404 Not Found (404)
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:              1770 / 1770, 100%
Elapsed time:        21.1s
2023/02/04 22:14:47 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Checks:              1770 / 1770, 100%
Elapsed time:        21.1s

2023/02/04 22:14:47 DEBUG : 4 go routines active
2023/02/04 22:14:47 Failed to copy: failed to open source object: pcloud error: 404 Not Found (404)

The files are visible and accessible via the web page, the windows app, and the linux console-client (pcloudcc) which has no trouble copying the files which rclone cannot access.

2

Can you identify the files not working and share the file names?

We've seen a few of these before as I'd imagine it might be an odd translate.

3

The filename is nothing special. No special characters, no unicode and it appears the same in all clients.

# rclone lsf "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow"|cat -v
1. Section Overview.srt
10. Exercise Solutions.srt
11. Section Review.srt
2. For Loop.srt
3. Nested Loops.mp4
3. Nested Loops.srt
4. Conditions, Break, & Continue.mp4
4. Conditions, Break, & Continue.srt
5. Documentation & Terminology.mp4
5. Documentation & Terminology.srt
6. Rune.srt
7. String Type.mp4
7. String Type.srt
8. Switch Statements.srt
9. If Statements.srt

# rclone -vv cat "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/7. String Type.srt"|wc -l
2023/02/05 08:23:32 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "-vv" "cat" "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/7. String Type.srt"]
2023/02/05 08:23:32 DEBUG : Creating backend with remote "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/7. String Type.srt"
2023/02/05 08:23:32 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/02/05 08:23:40 DEBUG : fs cache: adding new entry for parent of "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/7. String Type.srt", "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow"
2023/02/05 08:23:41 DEBUG : 1. Section Overview.srt: Excluded
2023/02/05 08:23:41 DEBUG : 10. Exercise Solutions.srt: Excluded
2023/02/05 08:23:41 DEBUG : 11. Section Review.srt: Excluded
2023/02/05 08:23:41 DEBUG : 2. For Loop.srt: Excluded
2023/02/05 08:23:41 DEBUG : 3. Nested Loops.srt: Excluded
2023/02/05 08:23:41 DEBUG : 3. Nested Loops.mp4: Excluded
2023/02/05 08:23:41 DEBUG : 4. Conditions, Break, & Continue.srt: Excluded
2023/02/05 08:23:41 DEBUG : 4. Conditions, Break, & Continue.mp4: Excluded
2023/02/05 08:23:41 DEBUG : 5. Documentation & Terminology.srt: Excluded
2023/02/05 08:23:41 DEBUG : 5. Documentation & Terminology.mp4: Excluded
2023/02/05 08:23:41 DEBUG : 6. Rune.srt: Excluded
2023/02/05 08:23:41 DEBUG : 7. String Type.mp4: Excluded
2023/02/05 08:23:41 DEBUG : 8. Switch Statements.srt: Excluded
2023/02/05 08:23:41 DEBUG : 9. If Statements.srt: Excluded
2023/02/05 08:23:42 DEBUG : Couldn't parse Date: from server c41.pcloud.com: "Sat, 04 Feb 2023 21:23:42 +0000": parsing time "Sat, 04 Feb 2023 21:23:42 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 04 Feb 2023 21:23:42 +0000" as " "
2023/02/05 08:23:42 DEBUG : 4 go routines active
607

# rclone -vv cat "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/6. Rune.srt"|wc -l
2023/02/05 08:24:01 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "-vv" "cat" "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/6. Rune.srt"]
2023/02/05 08:24:01 DEBUG : Creating backend with remote "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/6. Rune.srt"
2023/02/05 08:24:01 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/02/05 08:24:09 DEBUG : fs cache: adding new entry for parent of "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow/6. Rune.srt", "pcloud:Training/golang/Learn How To Code Google's Go (golang) Programming Language/37. Control Flow"
2023/02/05 08:24:09 DEBUG : 1. Section Overview.srt: Excluded
2023/02/05 08:24:09 DEBUG : 10. Exercise Solutions.srt: Excluded
2023/02/05 08:24:09 DEBUG : 11. Section Review.srt: Excluded
2023/02/05 08:24:09 DEBUG : 2. For Loop.srt: Excluded
2023/02/05 08:24:09 DEBUG : 3. Nested Loops.srt: Excluded
2023/02/05 08:24:09 DEBUG : 3. Nested Loops.mp4: Excluded
2023/02/05 08:24:09 DEBUG : 4. Conditions, Break, & Continue.srt: Excluded
2023/02/05 08:24:09 DEBUG : 4. Conditions, Break, & Continue.mp4: Excluded
2023/02/05 08:24:09 DEBUG : 5. Documentation & Terminology.srt: Excluded
2023/02/05 08:24:09 DEBUG : 5. Documentation & Terminology.mp4: Excluded
2023/02/05 08:24:09 DEBUG : 7. String Type.srt: Excluded
2023/02/05 08:24:09 DEBUG : 7. String Type.mp4: Excluded
2023/02/05 08:24:09 DEBUG : 8. Switch Statements.srt: Excluded
2023/02/05 08:24:09 DEBUG : 9. If Statements.srt: Excluded
2023/02/05 08:24:11 DEBUG : Couldn't parse Date: from server vc549.pcloud.com: "Sat, 04 Feb 2023 21:24:11 +0000": parsing time "Sat, 04 Feb 2023 21:24:11 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 04 Feb 2023 21:24:11 +0000" as " "
2023/02/05 08:24:11 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2023/02/05 08:24:11 ERROR : 6. Rune.srt: Failed to open: pcloud error: 404 Not Found (404)
2023/02/05 08:24:11 DEBUG : 4 go routines active
2023/02/05 08:24:11 Failed to cat: pcloud error: 404 Not Found (404)
0
4

I'm having a similar problem, in which a particular image file on pCloud is not found when I try to copy it to my local computer, generating a '404 Not Found' error. I'm using v1.61.1 of rclone on computers running linux debian 11.6.

$ ls testdir
img_0513s.jpg
$ file testdir/img_0513s.jpg
testdir/img_0513s.jpg: JPEG image data, JFIF standard 1.01, resolution (DPI), density 72x72, segment length 16, baseline, precision 8, 720x540, components 3
$ rclone copy testdir remote:testdir
$ rclone lsl remote:testdir
400659 2004-09-03 09:19:46.000000000 img_0513s.jpg

$ rclone copy remote:testdir/img_0513s.jpg . -vv --retries=1
2023/02/04 13:36:12 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "copy" "remote:testdir/img_0513s.jpg" "." "-vv" "--retries=1"]
2023/02/04 13:36:12 DEBUG : Creating backend with remote "remote:testdir/img_0513s.jpg"
2023/02/04 13:36:12 DEBUG : Using config file from "/home/paul/.config/rclone/rclone.conf"
2023/02/04 13:36:13 DEBUG : fs cache: adding new entry for parent of "remote:testdir/img_0513s.jpg", "remote:testdir"
2023/02/04 13:36:13 DEBUG : Creating backend with remote "."
2023/02/04 13:36:13 DEBUG : fs cache: renaming cache item "." to be canonical "/home/paul"
2023/02/04 13:36:13 DEBUG : img_0513s.jpg: Need to transfer - File not found at Destination
2023/02/04 13:36:14 DEBUG : Couldn't parse Date: from server vc549.pcloud.com: "Sat, 04 Feb 2023 21:36:13 +0000": parsing time "Sat, 04 Feb 2023 21:36:13 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 04 Feb 2023 21:36:13 +0000" as " "
2023/02/04 13:36:14 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2023/02/04 13:36:14 ERROR : img_0513s.jpg: Failed to copy: failed to open source object: pcloud error: 404 Not Found (404)
2023/02/04 13:36:14 ERROR : Attempt 1/1 failed with 1 errors and: failed to open source object: pcloud error: 404 Not Found (404)
2023/02/04 13:36:14 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 1.8s

2023/02/04 13:36:14 DEBUG : 4 go routines active
2023/02/04 13:36:14 Failed to copy: failed to open source object: pcloud error: 404 Not Found (404)

$ rclone version
rclone v1.61.1

os/version: debian 11.6 (64 bit)
os/kernel: 5.10.0-21-amd64 (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.19.4
go/linking: static
go/tags: none
$ rclone config show
[remote]
type = pcloud
hostname = api.pcloud.com

5

Couldn't parse Date: from server vc549.pcloud.com:
same error, same server

6

pcloud is definitely doing something fun in the background...

From windows I opened the file which rclone could not read into notepad++, and saved it as "new.txt" at the root of the drive.

Rclone cannot read "pcloud:new.txt", same 404 error.

I go back to notepad++ and add a single carriage return to the end of the file (essentially changing its checksum), and save, overwriting new.txt.

rclone can now read "pcloud:new.txt", and displays the contents, with the extra trailing newline.

I go back to notepad++, remove the extra carriage return and save the file.

rclone can no longer access the file..

Here is a log snipped using --dump-bodies to show the fileid has not changed, but the file
is unreadable in one instance, and readable in another. (After adding a single blank line to the end and saving)

2023/02/05 08:56:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:55 DEBUG : HTTP REQUEST (req 0xc000776300)
2023/02/05 08:56:55 DEBUG : GET /getfilelink?fileid=47760414130 HTTP/1.1
Host: api.pcloud.com
User-Agent: rclone/v1.61.1
Authorization: XXXX
Accept-Encoding: gzip

2023/02/05 08:56:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:55 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/05 08:56:55 DEBUG : HTTP RESPONSE (req 0xc000776300)
2023/02/05 08:56:55 DEBUG : HTTP/1.1 200 OK
Connection: close
Content-Length: 297
Cache-Control: private, max-age=0
Content-Type: application/json; charset=utf-8
Date: Sat, 04 Feb 2023 21:56:55 GMT
Etag: "9iaDk9xbiwuqqJC4GUzyt4kWu7SV"
Server: CloudHTTPd-API v1.1
Vary: Accept-Encoding

{
	"result": 0,
	"dwltag": "EWrkBtIXGsbqwJRIrieWO7",
	"hash": 16316769610243643454,
	"size": 9569,
	"expires": "Sun, 05 Feb 2023 03:56:55 +0000",
	"path": "\/dpZfeXcATZlCG29vZ2QpaZYZVAcyo7Zg0ZZ0uFZZYFlPG66rFf5X2tplefme38hR7kvy\/new.txt",
	"hosts": [
		"vc549.pcloud.com",
		"vc742.pcloud.com"
	]
}
2023/02/05 08:56:55 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/05 08:56:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:55 DEBUG : HTTP REQUEST (req 0xc00095c700)
2023/02/05 08:56:55 DEBUG : GET /dpZfeXcATZlCG29vZ2QpaZYZVAcyo7Zg0ZZ0uFZZYFlPG66rFf5X2tplefme38hR7kvy/new.txt HTTP/1.1
Host: vc549.pcloud.com
User-Agent: rclone/v1.61.1
Authorization: XXXX
Accept-Encoding: gzip

2023/02/05 08:56:55 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:56 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/05 08:56:56 DEBUG : HTTP RESPONSE (req 0xc00095c700)
2023/02/05 08:56:56 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Length: 161
Content-Type: text/html
Date: Sat, 04 Feb 2023 21:56:56 +0000
Server: CloudHTTPd v1.1

<html><head><title>404 Not Found</title></head><body>404 Not Found<br>/dpZfeXcATZlCG29vZ2QpaZYZVAcyo7Zg0ZZ0uFZZYFlPG66rFf5X2tplefme38hR7kvy/new.txt</body></html>
2023/02/05 08:56:56 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2023/02/05 08:56:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:36 DEBUG : HTTP REQUEST (req 0xc000275300)
2023/02/05 08:56:36 DEBUG : GET /getfilelink?fileid=47760414130 HTTP/1.1
Host: api.pcloud.com
User-Agent: rclone/v1.61.1
Authorization: XXXX
Accept-Encoding: gzip

2023/02/05 08:56:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/05 08:56:37 DEBUG : HTTP RESPONSE (req 0xc000275300)
2023/02/05 08:56:37 DEBUG : HTTP/1.1 200 OK
Connection: close
Content-Length: 294
Cache-Control: private, max-age=0
Content-Type: application/json; charset=utf-8
Date: Sat, 04 Feb 2023 21:56:37 GMT
Etag: "R8KpocllfHhoThtz6sml5hYt9VXk"
Server: CloudHTTPd-API v1.1
Vary: Accept-Encoding

{
	"result": 0,
	"dwltag": "xoD2l4Xx6m50fFtzEuN6YL",
	"hash": 10078123798631005975,
	"size": 9571,
	"expires": "Sun, 05 Feb 2023 03:56:37 +0000",
	"path": "\/dpZCHHvKbZlCG29vZ2QpaZYZEIcyo7Zg0ZZ0uFZZKKNg1xDWE0pDduuQi3bGjJ7LbXG7\/new.txt",
	"hosts": [
		"c54.pcloud.com",
		"c452.pcloud.com"
	]
}
2023/02/05 08:56:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/05 08:56:37 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:37 DEBUG : HTTP REQUEST (req 0xc0001d6400)
2023/02/05 08:56:37 DEBUG : GET /dpZCHHvKbZlCG29vZ2QpaZYZEIcyo7Zg0ZZ0uFZZKKNg1xDWE0pDduuQi3bGjJ7LbXG7/new.txt HTTP/1.1
Host: c54.pcloud.com
User-Agent: rclone/v1.61.1
Authorization: XXXX
Accept-Encoding: gzip

2023/02/05 08:56:37 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 08:56:38 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/05 08:56:38 DEBUG : HTTP RESPONSE (req 0xc0001d6400)
2023/02/05 08:56:38 DEBUG : HTTP/1.1 200 OK
Connection: close
Content-Length: 9571
Accept-Ranges: bytes
Content-Transfer-Encoding: binary
Content-Type: text/plain
Date: Sat, 04 Feb 2023 21:56:38 +0000
Etag: "1db06e7ddd80205206e2c150de63d5d497b86730"
Expires: Sun, 05 Feb 2023 03:56:37 +0000
Last-Modified: Sat, 23 Nov 2019 23:12:24 +0000
Server: CloudHTTPd v1.1

[contents of file here]

2023/02/05 08:56:38 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
7

Will rclone ever try the second host in this situation? It doesn't appear to.

8

might try hardcodiing. tho the docs mentioned it for oauth
--pcloud-hostname

9

I tried it just for fun. It overrides the api.pcloud.com with the new hostname which can't handle normal file info requests... so no go.

2023/02/05 09:33:16 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 09:33:16 DEBUG : HTTP REQUEST (req 0xc000668600)
2023/02/05 09:33:16 DEBUG : GET /listfolder?folderid=0 HTTP/1.1
Host: vc742.pcloud.com
User-Agent: rclone/v1.61.1
Authorization: XXXX
Accept-Encoding: gzip

2023/02/05 09:33:16 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/02/05 09:33:17 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/02/05 09:33:17 DEBUG : HTTP RESPONSE (req 0xc000668600)
2023/02/05 09:33:17 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Length: 106
Content-Type: text/html
Date: Sat, 04 Feb 2023 22:33:17 +0000
Server: CloudHTTPd v1.1

<html><head><title>404 Not Found</title></head><body>404 Not Found<br>/listfolder?folderid=0</body></html>
2023/02/05 09:33:17 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
10

forgot that i had a pcloud account,

might mean nothing but
my account uses evc as in evc68.pcloud.com - Luxembourg
the two other posters, use vc as in vc549.pcloud.com - Florida

notice that the file is downloaded successfully.

rclone copy pcloud01:zork/file.ext . -vv --retries=1 --config=c:\data\rclone\rclone.conf  
2023/02/04 19:29:30 DEBUG : rclone: Version "v1.61.1" starting with parameters ["d:\\data\\rclone\\exe\\windows\\rclone-v1.61.1-windows-amd64\\rclone_1.61.1.exe" "copy" "pcloud01:zork/file.ext" "." "-vv" "--retries=1" "--config=c:\\data\\rclone\\rclone.conf"]
2023/02/04 19:29:30 DEBUG : Creating backend with remote "pcloud01:zork/file.ext"
2023/02/04 19:29:30 DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
2023/02/04 19:29:32 DEBUG : fs cache: adding new entry for parent of "pcloud01:zork/file.ext", "pcloud01:zork"
2023/02/04 19:29:32 DEBUG : Creating backend with remote "."
2023/02/04 19:29:32 DEBUG : fs cache: renaming cache item "." to be canonical "//?/C:/data/rclone/rr/other/newtest/minimal"
2023/02/04 19:29:33 DEBUG : file.ext: Need to transfer - File not found at Destination
2023/02/04 19:29:33 DEBUG : Couldn't parse Date: from server evc68.pcloud.com: "Sun, 05 Feb 2023 00:29:33 +0000": parsing time "Sun, 05 Feb 2023 00:29:33 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 05 Feb 2023 00:29:33 +0000" as " "
2023/02/04 19:29:34 DEBUG : file.ext: sha1 = da39a3ee5e6b4b0d3255bfef95601890afd80709 OK
2023/02/04 19:29:34 INFO  : file.ext: Copied (new)
2023/02/04 19:29:34 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         3.3s
11

did more testing, seems the issue could be with servers in the US region.

tested v1.53.0 and that behaves same as v1.61.1

my pcloud server is in EU region - rclone copy downloads the file.
using OP's pcloud server in US region - rclone copy failed with 404

rclone copy pcloud01:zork . --pcloud-hostname="vc549.pcloud.com" -vv --retries=1 --config=c:\data\rclone\rclone.conf  
2023/02/04 20:19:00 DEBUG : rclone: Version "v1.61.1" starting with parameters ["d:\\data\\rclone\\exe\\windows\\rclone-v1.61.1-windows-amd64\\rclone_1.61.1.exe" "copy" "pcloud01:zork" "." "--pcloud-hostname=vc549.pcloud.com" "-vv" "--retries=1" "--config=c:\\data\\rclone\\rclone.conf"]
2023/02/04 20:19:00 DEBUG : Creating backend with remote "pcloud01:zork"
2023/02/04 20:19:00 DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
2023/02/04 20:19:00 DEBUG : pcloud01: detected overridden config - adding "{vOdwA}" suffix to name
2023/02/04 20:19:00 DEBUG : Couldn't parse Date: from server vc549.pcloud.com: "Sun, 05 Feb 2023 01:19:00 +0000": parsing time "Sun, 05 Feb 2023 01:19:00 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 05 Feb 2023 01:19:00 +0000" as " "
2023/02/04 20:19:00 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2023/02/04 20:19:00 DEBUG : Couldn't parse Date: from server vc549.pcloud.com: "Sun, 05 Feb 2023 01:19:00 +0000": parsing time "Sun, 05 Feb 2023 01:19:00 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 05 Feb 2023 01:19:00 +0000" as " "
2023/02/04 20:19:00 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2023/02/04 20:19:00 Failed to create file system for "pcloud01:zork": couldn't list files: pcloud error: 404 Not Found (404)
12

This does look like a pcloud problem - Has anyone tried contacting them?

Rclone just tries the first host.

It could rotate them on retries.

However pcloud is returning a 404 error which means definitively - this file is not found - that isn't an error rclone would normally retry.

pcloud also didn't return the error JSON it should do which indictates to me this is a problem.

Here are the docs on the API method rclone is using

https://docs.pcloud.com/methods/streaming/getfilelink.html

It says

On success it will return array hosts with servers that have the file. The first server is the one we consider best for current download.

So I think rclone is doing the right thing choosing the first one.

13

Thanks @ncw,

I did a manual test, calling listfolder and getfilelink, then tried each of the hosts. The first (vc549.pcloud.com) did not work (as expected) and the second (vc705.pcloud.com) did return the file.

The document you linked to above indicates this is not how its meant to work...

So, this issue is clearly on pCloud's end. I'll raise a ticket there and see what they say (experience says the response will be unsatisfactory at best...)

EDIT: I just realised the result from "getfilelink" does not need authentication to download. Hence pasting the link here gives everyone access to my files (at least for a short time) :confounded:

EDIT2: I tried another file which was failing, and while the first host returned 404, the second host returned ok, with a valid content length, then return no data. Hence forcing rclone to iterate through all hosts would not be a solution. Again, this file works fine via the windows and linux pcloud clients.

For the time being, I'm using rclone for the bulk download, then rsync via the pcloud client to pick up the missing files. (pcloud client is incredibly slow, and rsync doesn't hold a candle to rclones multi-file transfer abilities)

14

Great testing :slight_smile:

It might be that there is a better API for rclone to be using. I saw the file_ops API

https://docs.pcloud.com/methods/fileops/file_open.html

Maybe that would be more reliable - I don't know.

15

Wait, what? Open/Close/Read/Write/Seek? Sounds too good to be true.

A Quick test on the first file which was returning 404 :-

$ curl -s -H 'Authorization: XXXX' 'api.pcloud.com/stat?path=/new.txt'
{
        "result": 0,
        "metadata": {
                "name": "new.txt",
                "created": "Sat, 02 Apr 2022 07:51:00 +0000",
                "thumb": false,
                "modified": "Sat, 02 Apr 2022 07:51:00 +0000",
                "isfolder": false,
                "fileid": 47789346407,
                "hash": 16316769610243643454,
                "comments": 0,
                "category": 4,
                "id": "f47789346407",
                "isshared": false,
                "ismine": true,
                "size": 9569,
                "parentfolderid": 0,
                "contenttype": "text\/plain",
                "icon": "document"
        }
}
$ curl -s -H 'Connection: keep-alive' -H 'Authorization: XXXX' 'https://api.pcloud.com/file_open?fileid=47789346407&flags=0' 'https://api.pcloud.com/file_read?fd=1&count=9569'
{
        "result": 0,
        "fd": 1,
        "fileid": 47789346407
}
[complete file contents]...

Open was successful, and the complete file contents were returned... So maybe its a way forward?

However, when I tested the second case (which returned 404 from the first host and no data from the second), and it continued to return no data..

> GET /file_read?fd=1&count=144993 HTTP/1.1
> Host: api.pcloud.com
> Accept: */*
> Connection: keep-alive
> Authorization: XXXX
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: CloudHTTPd-API v1.1
< Date: Mon, 06 Feb 2023 11:16:53 GMT
< Content-Type: application/octet-stream
< Content-Length: 144993
< Cache-Control: private, max-age=0
< Vary: Accept-Encoding
< Connection: keep-alive
< Keep-Alive: timeout=1800
<
* Failed writing body (0 != 1432)
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):
16

:slight_smile:

:frowning:

It wouldn't be difficult to switch rclone to the fileops methods, but I don't think its worth doing unless it fixes the problem.

17

Hi @ncw,

So the support response was

We would like to inform you that we do not provide dedicated technical support for the API at this time. You can find all documentation at https://docs.pcloud.com/

regardless of this, the files which were causing issues appear to have been migrated to a new host, and are working now.

{
    ...
    "hosts": [
        "vc742.pcloud.com",
        "vc705.pcloud.com"
    ],
}

Thank you very much for your time helping me with this.

18

That is not entirely satisfactory as you weren't trying for technical support, you were trying to report an issue with the API. Oh well!

Great news.

19

My problem has resolved, too. I can now download the problem files without issues. Maybe because of a change of servers, from vc549.pcloud.com to:

"hosts": [
"c457.pcloud.com",
"c382.pcloud.com"
]

20

I got this from pCloud support this morning

Hello,

Thank you for the email.

I've managed to confirm there is an issue with vc549.pcloud.com. Does the system still try to use that server in your tries to download the link?

Should you require any assistance, do not hesitate to contact us.

Regards,
Chris
pCloud Support Team

So at least they confirmed there were issues.

1 Like