Rclone with multiple ceph radosgw with roundrobin DNS

Help and Support
1

Hello,

I have a ceph cluster with 6 radosgw with a DNS entry round-robin over those 6 servers. When I start a transfert, rclone does only one dns request and all the tranfert go on one gateway instead of spread requests over all gateways. Is there a solution to allow rclone to do a dns request before each transfer ?

Best regards,

What is your rclone version (eg output from rclone -V)
rclone v1.34

Which OS you are using and how many bits (eg Windows 7, 64 bit)
Ubuntu 14.04.3 LTS
Kernel 4.4.0-34-generic

Which cloud storage system are you using? (eg Google Drive)
Ceph Jewel 10.2.3

The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone copy /tmp remote:tmp

Yoann

2

I thought rclone did this, but apparently not…

Will do a bit more research.

3

Actually I think go does do DNS round robin according to my tests.

However does the server you are accessing it from share a subnet with one of the round robin DNS entries - if it does then go will favour that one - that is RFC 3484 in action.

This was fixed in go 1.6 as far as I can see: https://github.com/golang/go/issues/13283

4

both client and radosgw servers are on the same subnet (/24)

rclone is “statically linked” so, even if I update go on my server, that shouldn’t change this behaviour, should it ?

5

That explains it then. rclone will use longest common prefix length which may just be one IP (see RFC 3484)

Upgrading go on your server will make no difference. The bug I linked to is fixed already and is in go1.7 which is what I used to make the downloadable packages.

You could try selecting the C resolver instead of the Go resolver Eg https://golang.org/pkg/net/#hdr-Name_Resolution - You’d need to compile from source to use the C resolver. However I don’t think it will make any difference - at least it didn’t in my tests.