In an earlier blog I wrote that SSH keys need to be managed. It’s important!
In the comments I was asked about keytabs. I felt this such a good question that it deserved it’s own blog entry.
The basics of Kerberos auth
In essence, kerberos is a “ticket based” authentication scheme. You start by requesting a TGT (“Ticket granting ticket”). This, typically, is where you enter your password. This TGT is then used to request service tickets to access resources. There’s a description which goes into a lot more detail, but is pretty readable.
Now all of this is pretty invisible and runs under the covers; the user never needs to see any of this. Indeed, Windows Active Directory is kerberos based!
So let’s see this in action:
$ ssh sweh@kclient
sweh@kclient's password:
kclient$
So far that looks just the same as any other login! But if we look deeper…
kclient$ klist
Ticket cache: FILE:/tmp/krb5cc_500_WfoXRC1791
Default principal: sweh@SPUDDY.ORG
Valid starting Expires Service principal
09/24/16 18:46:08 09/25/16 18:46:08 krbtgt/SPUDDY.ORG@SPUDDY.ORG
renew until 09/24/16 18:46:08
I have a TGT associated with my login. That was all transparent. And I can use this to login to other machines…
kclient$ ssh kdc
kdc$ exit
Connection to kdc closed.
kclient$ klist
Ticket cache: FILE:/tmp/krb5cc_500_WfoXRC1791
Default principal: sweh@SPUDDY.ORG
Valid starting Expires Service principal
09/24/16 18:46:08 09/25/16 18:46:08 krbtgt/SPUDDY.ORG@SPUDDY.ORG
renew until 09/24/16 18:46:08
09/24/16 18:47:56 09/25/16 18:46:08 host/kdc.spuddy.org@SPUDDY.ORG
renew until 09/24/16 18:46:08
Well, it didn’t ask me for my password… and now I can see there’s a service ticket for that host.
So Kerberos can give us a form of “single sign on” (I only had to enter my password the once), and it’s all nice and transparent.
What is a keytab?
Everything so far has depended on being able to enter my password to
get that initial TGT. This is fine for humans, but for automated processes
(e.g. oracle wanting to scp files between servers) then it’s not very
useful.
So we have the ability to use a keytab file to get that initial key.
In my test environment I created a user1 and extracted that user’s
key into a keytab file:
$ kadmin -p sweh/admin
Authenticating as principal sweh/admin with password.
Password for sweh/admin@SPUDDY.ORG:
kadmin: ktadd -k user1.keytab user1@SPUDDY.ORG
...lines deleted...
kadmin: exit
$ ls -l user1.keytab
-rw------- 1 sweh sweh 666 Sep 24 18:53 user1.keytab
Now we can use this to login to kerberos:
$ kinit -k -t user1.keytab user1@SPUDDY.ORG
$ klist
Ticket cache: FILE:/tmp/krb5cc_500_WfoXRC1791
Default principal: user1@SPUDDY.ORG
Valid starting Expires Service principal
09/24/16 18:55:29 09/25/16 18:55:29 krbtgt/SPUDDY.ORG@SPUDDY.ORG
renew until 09/24/16 18:55:29
We can see that we didn’t get asked for a password, and the “Default principal” line mentions user1. As far as the kerberos infrastructure is concerned, I now am user1 and can use this to ticket to login as him:
$ ssh user1@kclient
Last login: Sat Sep 24 18:56:39 2016 from kclient.spuddy.org
[user1@kclient ~]$ exit
logout
Connection to kclient closed.
$ klist
Ticket cache: FILE:/tmp/krb5cc_500_WfoXRC1791
Default principal: user1@SPUDDY.ORG
Valid starting Expires Service principal
09/24/16 18:55:29 09/25/16 18:55:29 krbtgt/SPUDDY.ORG@SPUDDY.ORG
renew until 09/24/16 18:55:29
09/24/16 18:56:39 09/25/16 18:55:29 host/kclient.spuddy.org@SPUDDY.ORG
renew until 09/24/16 18:55:29
So we can see that these keytab files are as powerful as a passphrase-less ssh private key.
In some respects they are more powerful than ssh keys, because there’s no server side restrictions that can be applied; with ssh public key authentication the public key needs to be placed on the host for it to be usable, and we can limit where the private key can be used (“from=”) and what it can do (“command=”). With kerberos authentication we can login to any server where this ticket is valid (pretty much anywhere the account exists) and get a shell.
So it becomes critically important that keytab entries of this type are closely managed. Maybe they should be put into an equivalent of a password vault and only pulled “as needed” by the application? Or, at least, placed into a locked directory where no one except the app can reach it.
This is one reason why NIST recommends SSH keys over kerberos in many cases.
Service keytabs
There’s a second form of keytab file; these are service entries and are
typically of the form service/FQDN@domain. Your kerberos host probably
already has one in /etc/krb5.keytab for the host service
-rw------- 1 root root 448 Sep 24 18:21 /etc/krb5.keytab
You may have noticed the host/kclient.spuddy.org entries in the klist
entries above… that’s as a result of this type of service.
If you want to do kerberos authentication for web sites you would
create a HTTP service principal and then that keytab file would be
used by the web server to verify incoming requests.
Now these keytab entries are used to verify the service you’re talking to is the service you expect to talk to. In a way they are comparable to the ssh host keys or SSL certificate, but each service has their own key. Because of this identity assertion these keytab files should also be carefully managed.
k5login and k5users files
There’s one final “gotcha” with kerberos; it allows one user to trust
another. In $HOME/.k5login you can list other kerberos principals that
are allowed to login to this account.
So user2 has the following entry:
$ sudo cat ~user2/.k5login
user1@SPUDDY.ORG
Now this means that if I have a user1 TGT I can login as user2! We’ll
re-use the keytab file from earlier…
$ kinit -k -t user1.keytab user1@SPUDDY.ORG
$ ssh user2@kclient
Last login: Sat Sep 24 18:25:45 2016 from kclient.spuddy.org
[user2@kclient ~]$
In this respect the .k5login file is close to the ssh authorized_keys
file; if I could add my login to ~oracle/.k5login then I can login
as oracle without needing a password.
And, of course, we have NFS weaknesses if the user home directory is on an NFS server…
Worse, this file also works with ksu…
$ kinit -k -t user1.keytab user1@SPUDDY.ORG
$ ksu root
Authenticated user1@SPUDDY.ORG
Account root: authorization for user1@SPUDDY.ORG successful
Changing uid to root (0)
[root@kclient sweh]# cat /root/.k5login
user1@SPUDDY.ORG
Also note that ksu will also work with .k5users, which is similar
but allows for restrictions on the command. Do a man ksu for
details.
So just as how with ssh you need to set the AuthorizedKeysFile file
entry, in /etc/krb5.conf you should set k5login_directory directory
to something users can not modify themselves, and then have a management
tool to control them, just as you would control ssh public keys.
I’m not sure this is fully supported, though. I’ve had marginal results
(ssh works properly, ksu doesn’t?). So you may want to verify this
change actually works the way you want and if it doesn’t then create some
form of detective control to see if a .k5login or .k5users file has
been created.
Summary
Unless you have a specific need for kerberos (eg a kerberised file system where you need a ticket to access data), I’m not a fan.
We can see that for “service accounts” we have a keytab management problem to be resolved.
For human accounts there’s a tendancy to use Active Directory as the source of kerberos tickets, but I’ve previously written why I think this is a bad idea.
But if you do want to go down the kerberos path (and it may be reasonable for application servers, such as web servers) then we need to manage keytabs.
If you’re looking at a key management solution (e.g. one that manages SSL certs) then it’s worth seeing if it can also manage SSH keys and kerberos keytabs. It’s almost the same problem…