Serve ftp vs vsftpd


#1

I realize the products are very different but figured i’d toss a discussion thread about CPU consumption of the different products for comparison to see if there is anything that can be done to optimize the ftp serve backend.

Currently 2 upload streams on my VPS use about 20% of the a CPU to upload date with ftp serve. When compared to vsftpd (a dedicated product) it is more than double. The vsftpd daemon uses about 6%. Thats a considerable difference that I thought was worth mentioning.

Any ideas why rclone uses so much more CPU doing the same network work? Any ideas on how to optimize that from either a flag perspective or code?

Mine is just a basic command for now with mostly defaults:
rclone -vvv serve ftp local:/data/xxxx --addr :NNNN --user xxxx --pass xxxx --checkers 10 --transfers 10 --gid $gid --uid $uid --umask $UMASK --dir-cache-time 0


#2

Are they both uploading to local disk?

And is the transfer rate the same?

If you run rclone with the --rc flag then you can use the go pprof tools to find out what is using the most CPU - that would be a great first start.

I can’t think of anything obvious… You could try experimenting with --buffer-size - the default is 16M so you could try 0 and 128M say.

Note that --dir-cache-time 0 may use more CPU so you could try setting that to 1s to allow a little bit of caching.


#3

Yes the rate is very similar on each at about 8-10Mb an d both are local disk to the same location.

The --buffer-size seems to have dropped the cpu a bit to 10-14% on average so that is a good start.

I’ll take a look at this and see if I can get more detail.