Dial tcp: lookup api.pcloud.com: no such host

Help and Support
1

Ugh back to this seemingly DNS issue - problem this time is I do have DNS set to google so IDK whats going on now ?

we went through this over here Dedupe --by-hash throws error

PS C:\Users\Tony> rclone dedupe --by-hash  --dedupe-mode newest remote: --dry-run -vv --max-age "$lastrun" --filter-from "P:\scripts\filter-file.txt"
2022/11/04 21:06:44 DEBUG : --max-age 1.1293226524326774M to 2022-10-02 00:00:00 -0400 EDT m=-2927204.246697999
2022/11/04 21:06:44 DEBUG : rclone: Version "v1.60.0" starting with parameters ["C:\\Program Files (x86)\\rclone\\rclone.exe" "dedupe" "--by-hash" "--dedupe-mode" "newest" "remote:" "--dry-run" "-vv" "--max-age" "2022-10-02" "--filter-from" "P:\\scripts\\filter-file.txt"]
2022/11/04 21:06:44 DEBUG : Creating backend with remote "remote:"
2022/11/04 21:06:44 DEBUG : Using config file from "C:\\Users\\Tony\\.config\\rclone\\rclone.conf"
2022/11/04 21:06:44 INFO  : pcloud root '': Looking for duplicate md5 hashes using newest mode.
2022/11/05 11:41:35 DEBUG : pacer: low level retry 1/10 (error Get "https://api.pcloud.com/checksumfile?fileid=16962406568": read tcp 10.0.0.139:57189->74.120.9.121:443: i/o timeout)
2022/11/05 11:41:35 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2022/11/05 11:41:35 DEBUG : pacer: Reducing sleep to 15ms
2022/11/05 11:41:35 ERROR : My Pictures/2016/04/2016_04_22_52 (1).jpg: Failed to hash: failed to get hash: Get "https://api.pcloud.com/checksumfile?fileid=16962406568": dial tcp: lookup api.pcloud.com: no such host
2022/11/05 11:41:35 DEBUG : pacer: Reducing sleep to 11.25ms
2022/11/05 11:41:35 ERROR : My Pictures/2016/04/2016_04_22_52.jpg: Failed to hash: failed to get hash: Get "https://api.pcloud.com/checksumfile?fileid=11751825707": dial tcp: lookup api.pcloud.com: no such host
2022/11/05 11:41:35 DEBUG : pacer: Reducing sleep to 10ms
2022/11/05 11:41:35 ERROR : My Pictures/2016/04/2016_04_22_53 (1).jpg: Failed to hash: failed to get hash: Get "https://api.pcloud.com/checksumfile?fileid=16962408259": dial tcp: lookup api.pcloud.com: no such host
2022/11/05 11:41:35 ERROR : My Pictures/2016/04/2016_04_22_53.jpg: Failed to hash: failed to get hash: Get "https://api.pcloud.com/checksumfile?fileid=11751826292": dial tcp: lookup api.pcloud.com: no such host
2022/11/05 11:41:35 ERROR : My Pictures/2016/04/2016_04_22_54 (1).jpg: Failed to hash: failed to get hash: Get "https://api.pcloud.com/checksumfile?fileid=16962428078": dial tcp: lookup api.pcloud.com: no such host
2022/11/05 11:41:35 ERROR : My Pictures/2016/04/2016_04_22_54.jpg: Failed to hash: failed to get hash: Get "https://api.pcloud.com/checksumfile?fileid=11751827254": dial tcp: lookup api.pcloud.com: no such host
2

You'd want to figure out what's going on via that server/computer as that's just a DNS again. Nothing rclone related unfortunately.

3

but if nslookup on the same box always resolves how is this not an Rclone problem? is there any debug to see more detail??

4

https://www.nslookup.io/domains/api.pcloud.com/dns-records/#google

resolves across all the major DNS providers too

5

Rclone just calls the OS for a host name.

Guessing you are on Windows with some configuration perhaps blocking it? AV? Defender?

That's a general network error that you had a connection timeout to the provider.

That means DNS stopped working for rclone which could be AV/Defender/etc.

Rclone is just reporting the error it sees.

6

yes windows 10 - I don't have any third party AV/ad blocker etc just Windows Defender running which I can't seem to find any issues with

7

Right.

If no other person is reporting it, it's very unlikely rclone has a systemic DNS problem as it's more likely a local issue on your machine somewhere.

I'm not a Windows user so really can't offer much more to troubleshoot.

8

can you open https://api.pcloud.com in a web browser?

9

Hi tb582,

I don't think it is a DNS issue, it looks more like you are having transient network connectivity issues, that also sometimes happens to affect the connection to your DNS server.

Here is an observation supporting this (a timeout between two IP addresses is pure connectivity - no DNS involved):

Next you see this (which could be a DNS issue or a general connectivity issue):

which I would consider a general connectivity issue when seen immediately after the first issue.

Here are some posts with my suggestions to troubleshoot network issues like these on Windows:
https://forum.rclone.org/t/backblaze-b2-api-version/33834/7
https://forum.rclone.org/t/backblaze-b2-api-version/33834/13

1 Like
10

Not nitpick but DNS is already resolved to an IP as that's just a network/connectivity timeout so that would be a general connectivity issue.

11

Fully agree, as I (tried to) communicate above it:

12

Running ping -t api.pcloud.com I see no issues thus far running this - yet while that same ping -t is running fine doing the rclone dedup command hangs and eventually throws the same errors as above - again meanwhile ping -t is progressing just fine

13

OK, that was unexpected.

I still suspect some transient connectivity issues, but it seems more subtle than my experience.

Do you see the same with other rclone commands to pcloud?
Do you see the same with other rclone remotes (e.g. Google Drive)?

14

I honestly only use rclone with pcloud - and its seemingly more of an issue with dedupe than it is with other things lsf and copy commands seem fine.

BUT I think part of the issue might be the dedupe command and how it actually runs b/c it waits at this stage for a LONG time before it starts throwing errors


2022/11/07 13:40:07 DEBUG : rclone: Version "v1.60.0" starting with parameters ["C:\\Program Files (x86)\\rclone\\rclone.exe" "dedupe" "--by-hash" "--dedupe-mode" "newest" "remote:" "--dry-run" "-vv" "--max-age" "2022-10-02" "--filter-from" "P:\\scripts\\filter-file.txt"]
2022/11/07 13:40:07 DEBUG : Creating backend with remote "remote:"
2022/11/07 13:40:07 DEBUG : Using config file from "C:\\Users\\Tony\\.config\\rclone\\rclone.conf"
2022/11/07 13:40:07 INFO  : pcloud root '': Looking for duplicate md5 hashes using newest mode.
15

Good observation!

So rclone is doing a lot of network activity for a long time (I see 12+ hours in the first log).

It could be random and then you only see it with long running internet intensive commands after a random amount of time
Or it could be time dependent where it always happens after approximately the same amount of rclone execution time
Or it could be a certain hours of the day
Or something else

Which pattern do you see?

Are you using WiFi?

16

That seems maybe like a wonky home router or something dropping connections if it gets pushed too hard.

17

No - hard wired

Don't think its time of day nor hours -

here's the latest log run from today - thats over two hours from when it says its looking for duplicates vs actually showing ANY activity.

2022/11/07 13:40:07 INFO  : pcloud root '': Looking for duplicate md5 hashes using newest mode.
2022/11/07 15:57:05 DEBUG : pacer: low level retry 1/10 (error Get "https://api.pcloud.com/checksumfile?fileid=11749612966": dial tcp 74.120.9.235:443: connectex: A socket operation was attempted 
to an unreachable host.)
2022/11/07 15:57:05 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2022/11/07 15:57:05 DEBUG : pacer: Reducing sleep to 15ms
2022/11/07 15:57:05 ERROR : My Pictures/2015/06/2015_06_22_33.jpg: Failed to hash: failed to get hash: Get "https://api.pcloud.com/checksumfile?fileid=11749612966": dial tcp: lookup api.pcloud.com: no such host

meanwhile the ping running continuously during that same period!

Ping statistics for 74.120.9.235:
    Packets: Sent = 8736, Received = 8706, Lost = 30 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 43ms, Maximum = 796ms, Average = 58ms
Control-C

hmm just noticed the 30 lost packets

Reply from 74.120.9.235: bytes=32 time=85ms TTL=43
Reply from 74.120.9.235: bytes=32 time=53ms TTL=43
Reply from 74.120.9.235: bytes=32 time=69ms TTL=43
Reply from 74.120.9.235: bytes=32 time=47ms TTL=43
No resources.
No resources.
General failure.
General failure.
General failure.
Reply from 74.120.9.235: bytes=32 time=53ms TTL=43
Reply from 74.120.9.235: bytes=32 time=55ms TTL=43
Reply from 74.120.9.235: bytes=32 time=55ms TTL=43
Reply from 74.120.9.235: bytes=32 time=51ms TTL=43
Reply from 74.120.9.235: bytes=32 time=48ms TTL=43
Reply from 74.120.9.235: bytes=32 time=56ms TTL=43
Reply from 74.120.9.235: bytes=32 time=49ms TTL=43
Reply from 74.120.9.235: bytes=32 time=51ms TTL=43
Reply from 74.120.9.235: bytes=32 time=48ms TTL=43
Reply from 74.120.9.235: bytes=32 time=47ms TTL=43
Reply from 74.120.9.235: bytes=32 time=54ms TTL=43
Reply from 74.120.9.235: bytes=32 time=52ms TTL=43
Request timed out.
Reply from 74.120.9.235: bytes=32 time=53ms TTL=43
Reply from 74.120.9.235: bytes=32 time=58ms TTL=43

Let me turn on ping timestamps and re-run - see ya in a few hours :frowning: ---

I would like to figure out why it takes rclone hours to produce any verbose output - seems like it needs a super verbose (@ncw )

18

Seems like a bad network card/adapter/etc.

All the evidence points to a local isolated issue. Occam's Razor.

With ping, you've removed rclone from the equation.

19

I agree with @Animosity022, this pattern looks like a network adapter being reset by Windows, so you can probably find more info in the System Event Log seen in the Event Viewer. Good idea to add timestamps, that will make it easier.

I would disconnect/reconnect all network cables from your computer to the router to the wall and then restart router, switches and computer in that order. If this doesn't help then I would try updating all components and changing cables.

If you suspect an issue with rclone dedupe or pcloud, then I suggest you try reproducing on the smallest dataset possible. That is start with your smallest subfolder and then gradually move up in size until you see the issue.

You can add --dump headers if you would like some extra verbosity, it will show the communication between rclone and pcloud.

That seems to be correct behavior according to the code. First rclone collect a list of all files and then it collect the hashes for each of the files, that may take some time if you have a lot of data (or a slow/unstable connection):

Is the observed behavior different from your earlier dedupes or trials on smaller folders?

20

It would be easy to mark some checkers here then at least you'd see something going on in the stats. Worth doing?