How to make rclone slow? (FTP)

Help and Support
1

What is the problem you are having with rclone?

When using rclone sync with FTP, my hosting providers blocks my IP after some time. Very bad DDOS protection probably. Using --ftp-concurrency=1 fixed this problem. But now the DDOS protection seems to be more aggressive. How can I make rclone even slower? I did not find a way.

What is your rclone version (output from rclone version)

rclone v1.56.2

  • os/version: Microsoft Windows 10 Pro 2009 (64 bit)
  • os/kernel: 10.0.19043.1266 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.16.8
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

FTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync remote:public_html "U:\local_folder" -v -P --retries 1000000 --retries-sleep=15s --ftp-concurrency=1

The rclone config contents with secrets removed.

[redacted]
type = ftp
host = redacted
user = redacted
pass = redacted
tls = false
explicit_tls = false

A log from the command with the -vv flag

Messages like these after the IP got banned:

2021/10/09 16:54:35 Failed to create file system for "remote:public_html": NewFs: failed to make FTP connection to "IP_REDACTED:21": dial tcp IP_REDACTED:21: connectex: Ein Verbindungsversuch ist fehlgeschlagen, da die Gegenstelle nach einer bestimmten Zeitspanne nicht richtig reagiert hat, oder die hergestellte Verbindung war fehlerhaft, da der verbundene Host nicht reagiert hat.
2

not sure how to slow down rclone using ftp.
perhaps increase the amount of time rclone waits on a connection before a timeout error.
perhaps https://rclone.org/docs/#contimeout-time
or tweak https://rclone.org/ftp/#ftp-idle-timeout

3

I think that does not help, because it would only change how rclone handles dead connections. What I would need would be something like a sleep time after each check / transfer.

4

i translated the error into english and looks like a timeout issue.
so i that is why i suggested to increase the timeout.

"A connection attempt failed because the remote station did not react correctly after a certain period of time"

5

Yeah thats because of the IP ban. The connection does not get though so a timeout occurs. To make it more clear: More than 100000 checks are getting done. Then my IP gets blocked and nothing happens any longer (Only timeout error messages).

6

Hi Flo,

The above suggestions from @asdffdsa are best if your issue is related to slow down during the log in.

If I understand your situation correctly, then your issue is that you are being blocked after some time of too high activity.

If so, then try --tpslimit with a very small number, e.g. 0.1. If successful, then you can slowly try to increase until you are just under the limit.

PS: I would remove --retries 1000000, it will only make things worse by requiring more requests to transfer your files. You want to have the --tpslimit so low that you succeed in first try most of the time - and always succeed within 3 retries (the default if not specified).

1 Like
7

You can find some additional tips in the comments to this issue:

8

--tpslimit
--bwlimit

1 Like
9

Thank you very much, I didn't knew about tpslimit. I have just tried --tpslimit 1 with a fresh IP adress and I didn't got blocked. Perfect.

1 Like
10

thanks, i missed --tpslimit

1 Like
11

Hey.

Totally not what you’re asking but you should really, strongly, consider moving to a new host! FTP is outdated and insecure. The snippet of your config makes it seem like you’re also telling it not to encrypt.

You are just asking for trouble!

1 Like
12

Yes I know that FTP is bad and no encryption is even worse. But I had to do it like that because rclone says x509: cannot validate certificate for **IP REDACTED** because it doesn't contain any IP SANs when using explicit_tls = true. And SFTP (SSH) does not work with my host (They are using the popular cpanel). cpanel has backup functionality but I want to backup the files by syncing them to my local disk. And then the only way to do that seems to be FTP.

But because of your comment I tried again to get FTPS (explicit_tls = true) working with rclone. After some googling --ftp-no-check-certificate was the solution. Encrypted FTP without certificate checking should be better then no encryption...

Interesting: Filezilla has no problem at all connecting with this server by FTP and explicit TLS.

13

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.