Hello, I am using rclone “copy” to upload/download files to an AWS S3 bucket. One thing that I observed is that throughout the entire transfer process, multiple dns requests were issued and multiple TCP connections were opened to the IP addresses contained in those dns responses(answers). For single file (1G) upload, I found three different IP addresses (TCP connections). For directory (twenty 1G files) upload, I found more than ten TCP connections. My question (1) Is there any relationship between the number of TCP connections and the parameters(such as the file size, number of files)? (2) Is that number controlled by rclone(when and how many dns queries to send) or the provider (AwS S3)?
I’d expect to see roughly --checkers plus --transfers TCP connection if rclone is working at full parallelism. These default to 8 and 4 = 12. So more than 10 sounds reasonable!
You can control the number of TCP connections with --checkers and --transfers. The DNS queries are issued by the go standard library and are less under rclone’s control.
It might be if you compile rclone from source to make it use the system dns resolver it will behave differently with respect to DNS queries.