Errors in log, how to understand and fix them

Help and Support
1

Hi

Im runing rclone on a Synology NAS

Mainly its working fine but I have a few errors in my log, here is the errors:
Share has like 100 000 files, I get 860 errors in my log.

What can be the issue for these 8 Error types?

  1. couldn't list files: Get

2022/03/08 01:08:33 ERROR : Latvija/Price enquiry/2021.03.25 - FA - 1 - Faderåsen - Name Namexx/ventilation drawings: error reading destination directory: couldn't list files: Get

  1. dial tcp [2a0a:cc80:1::30]:443: connect: network is unreachable

2022/03/08 01:12:15 ERROR : Companyx/Prosjekter/Lursåsveien/Felles/ARK/2. Anbudstegninger/HUS D/A60 Skjema/A62 Vinduer/A62-002 Vinduskjema.pdf: Failed to copy: read metadata failed: Get "https://jfs.jottacloud.com/jfs/NAS/Jotta/Archive/Companyx/Companyx/Prosjekter/Lur̊sveien/Felles/ARK/2.%20Anbudstegninger/HUS%20D/A60%20Skjema/A62%20Vinduer/A62-002%20Vinduskjema.pdf": dial tcp [2a0a:cc80:1::30]:443: connect: network is unreachable

  1. connect: network is unreachable

2022/03/08 01:18:58 ERROR : Companyx/Prosjekt/Lureåsveien/Felles/ARK/2. Anbudstegninger/Lureåsveien_kjøkken.png: Failed to copy: Post "https://api.jottacloud.com/files/v1/allocate": dial tcp [2a0a:cc80:1::26]:443: connect: network is unreachable

  1. corrupted on transfer: sizes differ 97 vs 96

2022/03/08 01:09:20 ERROR : #recycle/Companyx/Prosjekt/Bråtenveien 6 - Client/KUNDE/Kontrakt/old/3b - Store Huset Endelige Planer_ Vindu Dører m mål.pdf.md5: corrupted on transfer: sizes differ 97 vs 96

  1. corrupted on transfer: sizes differ 114 vs 108

2022/03/08 01:10:54 ERROR : #recycle/Companyx/Prosjekt/Varden terrasse 1_959/Sharing/OLD, not in use/Noice calculation/Asplan viak støy/Uttalelse fra N kommune om gjenbruk støymåling fra 2005 - 2015.03.17.pdf.md5: corrupted on transfer: sizes differ 114 vs 108

  1. too many open files

2022/03/08 01:12:05 ERROR : Companyx/Prosjekter/Varden terrasse 1_959/Sharing PUBLIC - Be careful when use/Electro/2015.09.09 PDF elektro 4 tilvalg/Til kunder/Bakgrunn: failed to open directory "Companyx/Prosjekter/Varden terrasse 1_959/Sharing PUBLIC - Be careful when use/Electro/2015.09.09 PDF elektro 4 tilvalg/Til kunder/Bakgrunn": open /volume1/Companyx/Companyx/Prosjekter/Varden terrasse 1_959/Sharing PUBLIC - Be careful when use/Electro/2015.09.09 PDF elektro 4 tilvalg/Til kunder/Bakgrunn: too many open files

  1. NO ELABORATED ERROR MESSAGE!

2022/03/08 01:17:21 ERROR : Latvija/Price enquiry/2021.05.10 - B2 - 10 - Fjellsrude - Brannkonsult/Drawings/PDF/A40-05 Fasade Sorvest og Nordvest

  1. ","cause":"","error_id":"InvalidArgumentException","x-id":"723196909168"}"

hus 8-9.pdf: Failed to copy: HTTP error 400 (400 Bad Request) returned body: "{"code":400,"message":"Unable to validate decoded path: /modesty/Jotta/Archive/Companyx/Latvija/Price enquiry/### OLD/2021.05.10 - B2 - 10 - Fjellsrude - Brannkonsult/Drawings/PDF/A40-06 Fasade Nordøst og Sørøst\u2028hus 8-9.pdf","cause":"","error_id":"InvalidArgumentException","x-id":"723196909168"}"

2

looks like network issues.

and hard to share good advice, as none of the help and support questions were answered.....

3

What is the problem you are having with rclone?

1) 443: connect: network is unreachable

2022/03/10 02:10:44 ERROR : Download/Android ting/android-sdk/sources/android-19/com/android/internal/widget: error reading source directory: failed to open directory "Download/Android ting/android-sdk/sources/android-19/com/android/internal/widget": open /volume1/Dok/Download/Android ting/android-sdk/sources/android-19/com/android/internal/widget: too many open files
2022/03/10 02:10:44 ERROR : Huawei kopi av disker/e 06102021/Google Drive/Sketchup files and settings/PluginsPack/SketchUcation/Data/Rubyzip: error reading destination directory: couldn't list files: Get "https://jfs.jottacloud.com/jfs/modesty/Jotta/Archive/Dok/Huawei%20kopi%20av%20disker/e%2006102021/Google%20Drive/Sketchup%20files%20and%20settings/PluginsPack/SketchUcation/Data/Rubyzip": dial tcp [2a0a:cc80:1::30]:443: connect: network is unreachable

2) too many open files

2022/03/10 02:10:45 ERROR : Huawei kopi av disker/d 06102021/ArduinoData/packages/esp8266/hardware/esp8266/2.7.1/libraries/ESP8266HTTPClient/examples/DigestAuthorization: error reading source directory: failed to open directory "Huawei kopi av disker/d 06102021/ArduinoData/packages/esp8266/hardware/esp8266/2.7.1/libraries/ESP8266HTTPClient/examples/DigestAuthorization": open /volume1/Dok/Huawei kopi av disker/d 06102021/ArduinoData/packages/esp8266/hardware/esp8266/2.7.1/libraries/ESP8266HTTPClient/examples/DigestAuthorization: too many open files

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0

  • os/version: unknown
  • os/kernel: 4.4.180+ (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.2
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Jottacloud

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy /volume1/Dok/ NAS:Dok --no-unicode-normalization --progress --log-file=LogDok.txt

The rclone config contents with secrets removed.

 [NAS-1]
type = jottacloud
configVersion = 1
client_id = jottacli
client_secret =
tokenURL = https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token
token = {"access_token":""}
username = DonaldD
device = LENOVO
mountpoint = Summer

[NAS]
type = jottacloud
configVersion = 1
client_id = jottacli
client_secret =
tokenURL = https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token
token = {"access_token":""}
device =
mountpoint =

A log from the command with the -vv flag

2022/03/10 06:20:42 DEBUG : rclone: Version "v1.57.0" starting with parameters [ "rclone" "copy" /volume1/Dok/" "NAS:Dok" "--no-unicode-normalization" "--progress" "--log-file=LogDok.txt" "-vv"]

2022/03/10 06:20:42 DEBUG : Creating backend with remote "/volume1/surveilance/"
2022/03/10 06:20:42 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/03/10 06:20:42 DEBUG : Creating backend with remote "Nas:Dok"
2022/03/10 06:21:42 Failed to create file system for "NAS:Dok": couldn't get customer info: Get "https://api.jottacloud.com/account/v1/customer": dial tcp 185.179.130.26:443: i/o timeout
4

This is definitely some sort of networking problem.

It is using IPv6 here and the network was unavailable for some reason.

This happened when rclone tried to read a directory failed to open directory and can be caused by your system running out of file handles or a bug in rclone which is leaking file handles.

If it is a bug though, I'd expect it to get worse and worse which it doesn't appear to.

Can you set the ulimit on the Synology NAS? Or you could reduce --checkers to use less file handles.

Running lsof when rclone is running will tell you what file handles are in use

5

Thanks for answer

I run my normal rclone job in one ssh session and try lsof in another session, seems to be a unsupported command in synology.

image

It also strange that IP6 is used, my network is only IP4 from my pfsense router. (but maybe some clients also is trying IP6, maybe NAS also is trying)

I run the rclone command each day, but as seen under the ETA is long

image

q1: May a possible double run make these Errors?

q2: Is there some switches that can reduce copy time?

M

6

You could try using --bind X.Y.Z.Q to bind rclone to the IPv4 address on the machine it is running on - that will make sure it doesn't use IPv6.

If you run rclone again it should hopefuly complete properly.

This is likely caused by some networking problem. Maybe the --bind command will help.

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.