Rclone failing to authenticate to a kerberized Hadoop cluster

Help and Support
1

What is the problem you are having with rclone?

I am trying to read data from a Kerberized Hadoop cluster using rclone, but it is failing with no available namenodes error message. The root cause seems to be related to kerberos.

Run the command 'rclone version' and share the full output of the command.

rclone v1.60.1

  • os/version: redhat 7.9 (64 bit)
  • os/kernel: 3.10.0-1160.11.1.el7.x86_64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.3
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

I am using HDFS as a storage system.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

./rclone --config=rclone.conf ls -vv remote-hdfs:<path-to-my-file>

The rclone config contents with secrets removed.

[remote-hdfs]
type = hdfs
namenode = <host>:<port>
username = <my-username>
data_transfer_protection = authentication
service_principal_name = <my-principal-name>

A log from the command with the -vv flag

./rclone --config=rclone.conf ls -vv remote-hdfs:<path-to-my-file>
2022/11/17 18:32:57 DEBUG : rclone: Version "v1.60.1" starting with parameters ["./rclone" "--config=rclone.conf" "ls" "-vv" "remote-hdfs:<path-to-my-file>"]
2022/11/17 18:32:57 DEBUG : Creating backend with remote "remote-hdfs:<path-to-my-file>"
2022/11/17 18:32:57 DEBUG : Using config file from "/tmp/rclone.conf"
2022/11/17 18:32:57 Failed to create file system for "remote-hdfs:<path-to-my-file>": no available namenodes: SASL handshake: [Root cause: KDC_Error] KDC_Error: TGS Exchange Error: kerberos error response from KDC when requesting for <my-principal-name>: KRB Error: (7) KDC_ERR_S_PRINCIPAL_UNKNOWN Server not found in Kerberos database```
2

This seems to be the underlying error.

Take a look at this issue and see if anything in there helps: Getting KDC_ERR_S_PRINCIPAL_UNKNOWN error · Issue #367 · jcmturner/gokrb5 · GitHub

3

Thanks for your answer. I have checked the link you sent me.

The error message says that the KDC_ERR_S_PRINCIPAL_UNKNOWN. If I understand correctly, that means that the service principal name is not know. But in my case, I have specified it in my rclone conf file, also checked on my KDC server if that principal in in the principals list and it is exist.

Do you have any ideas what could be the issue ?

4

The error message seems to insist that your service principal name isn't in the list.

Triple check the spelling maybe?

I'm sorry I'm not a kerberos expert so I'm running out of ideas. If you can't get it to work then open a new issue on Github and I'll tag our HDFS backend developer who may be able to help (put a link to the forum issue in the issue).

5

Thanks again for your quick reply. I created a new issue on GitHub.
Here is the link: Rclone failing to authenticate to a kerberized Hadoop cluster · Issue #6581 · rclone/rclone · GitHub

1 Like
6

Hello again, I am writing to you so you don't waste your time solving my issue. It is fixed now :slight_smile:

The problem was I was specifying the user principal name and not the service principal name.

Thanks a lot for your support.

7

Whew! Glad you fixed it.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.