Avm / fritzbox configuration for rclone / restic

Hi experts,
I am considering to use a disk attached to an AVM Fritzbox (FritzOS 7.12) for use with restic backup.
I tried a configuration using ftp - failed partially. The backup worked but checking the storage resulted in errors such as "Error while Dialing fritz.box:21: 421 Unauthorized client limit reached."
I tried using webdav - failed as well. (I can provide details, if required).
Has anyone a working solution?

Dietrich

Are you using a username/password with the FTP?

How are you accessing the storage - with rclone serve restic or some other way?

Yes please!

In both cases logs with -vv would be useful.

Isn't that an error from the FTP server?

I would typically read that as your server not allowing anonymous logins. If that is the source of the error then configuring your FTP to either allow (more) anonymous logins (if that is your intention) or setting up a user & password with the right access permissions should solve it. I think something very similar can happen if you leave a lot of half-open connections running while repeatedly testing also - so be aware of that and check your connections in a firewall, increase the limits temporarily, or restart the server if you think it shouldn't be working but it isn't.

Fritzbox: this is a wlan router by AVM. I cannot really configure too much on it. FTP in principle works. It seems that ftp is not understanding all commands. Some work just fine:

$ restic -r rclone:fritzbox:TEST snapshots
enter password for repository:
repository 8436a127 opened successfully, password is correct
ID Time Host Tags Paths

7790eae7 2019-08-11 09:03:03 lapin18 /tmp/A
0180fe57 2019-08-11 09:05:44 lapin18 /etc/fstab

I can also recover e.g. /etc/fstab from the backup.
But:

$ rclone ls fritzbox:
2019/08/12 16:45:35 ERROR : : error listing: 500 'LIST ': command not understood.
2019/08/12 16:45:35 Failed to ls with 2 errors: last error was: 500 'LIST ': command not understood.
(paulus@lapin18)-(~)-(16:45:35): rclone -vv ls fritzbox:
2019/08/12 16:45:39 DEBUG : rclone: Version "v1.48.0" starting with parameters ["rclone" "-vv" "ls" "fritzbox:"]
2019/08/12 16:45:39 DEBUG : Using config file from "/home/paulus/.rclone.conf"
2019/08/12 16:45:39 DEBUG : ftp://fritz.box:21: Connecting to FTP server
2019/08/12 16:45:40 ERROR : : error listing: 500 'LIST ': command not understood.
2019/08/12 16:45:40 Failed to ls with 2 errors: last error was: 500 'LIST ': command not understood.

... to be continued (as I get the message that my post has too many links)

Dietrich

$ restic -r rclone:fritzbox:TEST -vv check
using temporary cache in /tmp/restic-check-cache-454157786
enter password for repository:
repository 8436a127 opened successfully, password is correct
created new cache in /tmp/restic-check-cache-454157786
create exclusive lock for repository
load indexes
check all packs
rclone: 2019/08/12 16:46:46 ERROR : ftp://fritz.box:21/TEST: Error while Dialing fritz.box:21: 421 Unauthorized client limit reached. Try again later.
rclone: 2019/08/12 16:46:46 ERROR : data/11: error listing: list: ftpConnection Dial: 421 Unauthorized client limit reached. Try again later.

which to me leads to the conclusion, that I should not use ftp access. sftp is not provided, unfortunately. On the other hand rclone cannot use telnet (which is provided for the fritzbox - o.k., I know, I should not use it, at least not in open environments).

FTP isn't a very well standardized protocol and there are many bad implementations.

Webdav is probably slightly better - I'd try that if I was you.

Actually, this was my first attempt before I tried ftp.

$ rclone -vv ls fritzdav:
2019/08/12 16:52:34 DEBUG : rclone: Version "v1.48.0" starting with parameters ["rclone" "-vv" "ls" "fritzdav:"]
2019/08/12 16:52:34 DEBUG : Using config file from "/home/user/.rclone.conf"
2019/08/12 16:52:34 ERROR : : error listing: couldn't list files: Propfind fritz.box/fritz.nas/: unsupported protocol scheme ""
2019/08/12 16:52:34 Failed to ls with 2 errors: last error was: couldn't list files: Propfind fritz.box/fritz.nas/: unsupported protocol scheme ""

fritz.box/fritz.nas can be mounted as cifs using the credentials given in to rclone.

I think that means you've left the http:// or https:// off the URL

Thanks, that was my fault.
But even with https: I get communication errors:
$ rclone ls fritzdav: --no-check-certificate
2019/08/14 13:29:41 ERROR : : error listing: couldn't list files: 400 Bad Request (ERR_INVALID_REQ)

400 Bad Request


ERR_INVALID_REQ
Webserver Wed, 14 Aug 2019 11:29:41 GMT: 400 Bad Request
2019/08/14 13:29:41 Failed to ls with 2 errors: last error was: couldn't list files: 400 Bad Request (ERR_INVALID_REQ)

400 Bad Request


ERR_INVALID_REQ
Webserver Wed, 14 Aug 2019 11:29:41 GMT: 400 Bad Request

Now the question is, why fritzbox does not respond to the webdav request. I tried several urls (as I found them in various descriptions for the device), but they seem to be wrong.
That's why I tried ftp.

May be I just cancel the idea or wait until the router supports better protocols such as sftp.

Thanks for your help anyhow

I just had a read of the docs for fritz box. It appears it supports using webdav to connect to an online storage provider. I'm not sure it supports serving webdav.

The recommended way of connecting seems to be via smb/CIFS.

So unless you want to use smb/CIFS looks like we are back to FTP.

To work around this limit try setting this to 1. If that works then you can try higher numbers.

  --ftp-concurrency int           Maximum number of FTP simultaneous connections, 0 for unlimited

I have set this count to 1: export RCLONE_FTP_CONCURRENCY=1
Great, now the backup works using ftp - at least for small backups.
I will now try all commands and larger backups and larger values for FTP_CONCURRENCY.

Thanks for your support!

Glad that is working! You can also set concurrency in the config file when you've found the right number.