[Freeipa-devel] PKINIT Handling in mixed/CA-less topologies

Post by Martin Babinsky
Hi List,
TL;DR we have to handle FAST channer establishment when KDC is not issued
PKINIT keypair
I have spent some time studying and fixing bugs/regressions caused by
incomplete consideration of PKINIT and anonymous principal setup regarding to
* replicas standed up against old (3.0.0) masters
* domain level 0 topologies
* CA-less deployments
I want to discuss the impact of these findings on existing functionality and
how to fix them so that 4.5.1 release will be more usable and free of subtle
but serious bugs (more on this later).
From conversation from Alexander and Simo it follows that anonymous PKINIT
feature is supposed to be used in domain level 1 deployments because only these
guarantee the presence of the features (CA ACLs and custom certificate
profiles) which allow for issuing certificates suitable for PKINIT
* on DL0 enforce no_pkinit on server/replica deployments
* during upgrade of DL0 deployments, do not issue PKINIT certificates
* during upgrade of DL1 deployments issue PKINIT certs
* extend ipa-server-certinstall to install/issue PKINIT certificates after
DL0/DL1 ugrade (have to be manually).
However, I found out that the only case when anonymous PKINIT actually works is
for fresh DL1 server install and upgrade and install of 4.5.0 replica against
4.5.0 master in DL1. The following use-cases either fail to install or leave
* setting up 4.5 replica against <4.5 master fails during anonymous
principal setup[1] (ticket states domain level 0, but DL1 is also
affected)
* setting up server-replica with `no_pkinit` option (CA-full or CA-less)
leaves the installation without non-working WebUI as anonymous PKINIT does
not work (ticket incoming)
* If we restrict DL0 installs to force no_pkinit[2] we will be left with
whole topologies where anonymous PKINIT does not work, so no WebUI auth
for them
We now have to decide how to properly support or avoid non-PKINIT deployments.
The current code which handles armoring of password auth requests[3] does not
actually work without PKINIT certificates, the fallback mechanism still fails
to obtain armor ccache[4].
I have concluded that for non-PKINIT cases we have
to use the old way to armor TGT request (i.e. establish fast channel by
kinit as service principal), but this means that the framewrok has to use a
service principal whose keytab it can read and use. After privilege separation,
however, we do not have direct access to HTTP keytab so how should we proceed
in this case? We definitely need to discuss this further.
Please state your suggestions and comments, and sorry for the long mail.

Thanks, Martin, for the thorough analysis.

I need to clarify *why* we need working Anonymous PKINIT. There are two
separate needs here:

- Enable clients with no access to a separate key to be usable for 2FA
accounts. This can be best explained as to support Kerberos auth from
non-enrolled machines or machines where no SSSD is in use. In such
cases we cannot use another credentials to create FAST channel and
pass 2FA creds with kinit.

- Enable IPA framework to perform password-based login for 2FA. With
privilege separation we don't have access to HTTP/... principal's
keytab anymore (gssproxy does) and neither GSSAPI nor gssproxy
support FAST channel wrapping for explicitly specified password+2FA
token.

For DL0 we do not officially support PKINIT, so first case is not
relevant. However, second case is what we need even on DL0 because
otherwise IPA framework does not work, as you have witnessed.

We thought that we could solve this problem by re-using anonymous
principal as 'normal' principal -- by fetching its keytab and
authenticating with the keys from it. But for anonymous principal MIT
Kerberos library does verification of the session key and requires it to
be provided with PKINIT PA DATA when there is no wrapping principal
keys.

See RFC 6112 section 4.1: https://tools.ietf.org/html/rfc6112#section-4.1

----
The Kerberos client can use the client's long-term keys, the client's
X.509 certificates [RFC4556], or any other pre-authentication data,
to authenticate to the KDC and requests an anonymous ticket in an AS
exchange where the client's identity is known to the KDC.

If the client in the AS request is anonymous, the anonymous KDC
option MUST be set in the request. Otherwise, the KDC MUST return a
KRB-ERROR message with the code KDC_ERR_BADOPTION.
----

Corresponding code in MIT Kerberos is this:
https://github.com/krb5/krb5/blob/master/src/lib/krb5/krb/get_in_tkt.c#L157

So, using keytab for anonymous principal does not work. We either can
have another principal to perform wrapping or actually fix PKINIT for
DL0 for the purpose of IPA framework.

The latter is easy to achieve. Certmonger maintains two local CAs:
SelfSign and 'local':

# getcert list-cas
[....]
CA 'SelfSign':
is-default: no
ca-type: INTERNAL:SELF
next-serial-number: 01
CA 'local':
is-default: no
ca-type: EXTERNAL
helper-location: /usr/libexec/certmonger/local-submit

The first one self-signs whatever request you provide, the second one
signs it with a locally generated CA which is unique to each host. The
latter one doesn't perform any checks and simply signs the request.

Obviously, relying on certmonger's local CA to provide PKINIT to other
IPA clients does not scale. But we already estblished we wouldn't do
that. In IPA framework which runs on the very same host as KDC, we can
have access to the same public key KDC would be using for itself and can
kinit with it as an anchor:

kinit -X x509_anchor=/path/to/local-ca.crt -n

This approach allows us to avoid any modification to /etc/krb5.conf on
IPA master. An IPA framework would only need to have access to the
public key of local CA. And local CA is something certmonger provides
since its first run.

Yes, we'll need to manage upgrades from DL0 to DL1 for PKINIT. In
practice this will mean we have to:

- replace local CA-issued KDC certificate if we were upgraded to become
IPA-managed CA

- replace local CA-issued KDC certificate with externally provided KDC
certificate if we were upgraded and provided with explicit certificates

This is certainly doable and primary benefit is that we wouldn't need to
have any fallbacks anymore. We would always use Anonymous PKINIT within
the IPA framework and be done with it.

--
/ Alexander Bokovoy
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Simo Sorce

2017-03-23 14:36:57 UTC

Thanks, Martin, for the thorough analysis.
I need to clarify *why* we need working Anonymous PKINIT. There are two
- Enable clients with no access to a separate key to be usable for 2FA
accounts. This can be best explained as to support Kerberos auth from
non-enrolled machines or machines where no SSSD is in use. In such
cases we cannot use another credentials to create FAST channel and
pass 2FA creds with kinit.
- Enable IPA framework to perform password-based login for 2FA. With
privilege separation we don't have access to HTTP/... principal's
keytab anymore (gssproxy does) and neither GSSAPI nor gssproxy
support FAST channel wrapping for explicitly specified password+2FA
token.
For DL0 we do not officially support PKINIT, so first case is not
relevant. However, second case is what we need even on DL0 because
otherwise IPA framework does not work, as you have witnessed.
We thought that we could solve this problem by re-using anonymous
principal as 'normal' principal -- by fetching its keytab and
authenticating with the keys from it. But for anonymous principal MIT
Kerberos library does verification of the session key and requires it to
be provided with PKINIT PA DATA when there is no wrapping principal
keys.
See RFC 6112 section 4.1: https://tools.ietf.org/html/rfc6112#section-4.1
----
The Kerberos client can use the client's long-term keys, the client's
X.509 certificates [RFC4556], or any other pre-authentication data,
to authenticate to the KDC and requests an anonymous ticket in an AS
exchange where the client's identity is known to the KDC.
If the client in the AS request is anonymous, the anonymous KDC
option MUST be set in the request. Otherwise, the KDC MUST return a
KRB-ERROR message with the code KDC_ERR_BADOPTION.
----
https://github.com/krb5/krb5/blob/master/src/lib/krb5/krb/get_in_tkt.c#L157
So, using keytab for anonymous principal does not work. We either can
have another principal to perform wrapping or actually fix PKINIT for
DL0 for the purpose of IPA framework.
# getcert list-cas
[....]
is-default: no
ca-type: INTERNAL:SELF
next-serial-number: 01
is-default: no
ca-type: EXTERNAL
helper-location: /usr/libexec/certmonger/local-submit
The first one self-signs whatever request you provide, the second one
signs it with a locally generated CA which is unique to each host. The
latter one doesn't perform any checks and simply signs the request.
Obviously, relying on certmonger's local CA to provide PKINIT to other
IPA clients does not scale. But we already estblished we wouldn't do
that. In IPA framework which runs on the very same host as KDC, we can
have access to the same public key KDC would be using for itself and can
kinit -X x509_anchor=/path/to/local-ca.crt -n
This approach allows us to avoid any modification to /etc/krb5.conf on
IPA master. An IPA framework would only need to have access to the
public key of local CA. And local CA is something certmonger provides
since its first run.
Yes, we'll need to manage upgrades from DL0 to DL1 for PKINIT. In
- replace local CA-issued KDC certificate if we were upgraded to become
IPA-managed CA
- replace local CA-issued KDC certificate with externally provided KDC
certificate if we were upgraded and provided with explicit certificates
This is certainly doable and primary benefit is that we wouldn't need to
have any fallbacks anymore. We would always use Anonymous PKINIT within
the IPA framework and be done with it.

Just to recap the reason to support PKINIT locally is for supporting 2FA
in the WebUI which is a hard requirement.
Using the local CA will make this work for local logins ONLY and this is
OK.
The kinit command should really be called with this option:
x509_anchor=/var/kerberos/krb5kdc/cacert.pem so it will always work
regardless of what CA cert is there (the local one or the real CA one).

The only issue will be handling SELinux issues, if those are a problem
we can also simply copy the local-ca.crt
in /var/lib/ipa/api/pkinit-ca.crt and always call kinit with that file.

We would need to make sure we copy there the correct CA cert for the job
(certmonger's local-ca crt if no pkinit is enabled or whatever CA cert
we are given if pkinit is enabled).

Simo.

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Alexander Bokovoy

2017-03-23 14:46:20 UTC

Just to recap the reason to support PKINIT locally is for supporting 2FA
in the WebUI which is a hard requirement.

Correct.

Post by Simo Sorce
Using the local CA will make this work for local logins ONLY and this is
OK.
x509_anchor=/var/kerberos/krb5kdc/cacert.pem so it will always work
regardless of what CA cert is there (the local one or the real CA one).

The option is -X x509_anchor=...

Post by Simo Sorce
The only issue will be handling SELinux issues, if those are a problem
we can also simply copy the local-ca.crt
in /var/lib/ipa/api/pkinit-ca.crt and always call kinit with that file.

I think we can do that in pre-start hook in httpd.service.d/ipa.conf

Post by Simo Sorce
We would need to make sure we copy there the correct CA cert for the job
(certmonger's local-ca crt if no pkinit is enabled or whatever CA cert
we are given if pkinit is enabled).

Martin Babinsky

2017-03-24 08:43:41 UTC

Just to recap the reason to support PKINIT locally is for supporting 2FA
in the WebUI which is a hard requirement.

Correct.

The option is -X x509_anchor=...

I think we can do that in pre-start hook in httpd.service.d/ipa.conf

--
/ Alexander Bokovoy

Ok so let's forget for a moment about the clients and focus on framework and
PKINIT. We then have two options how to handle FAST wrapping:

1.) use a custom principal for FAST channel, e.g. ipaapi/`hostname`
This is analogous to how were things done before privilege
separation/PKINIT work. A plus is that you configure it once and it
works regardless of whether PKINIT is configured properly or not. A
minus is that when PKINIT is configured you are still using
one special mechanism for FAST/2FA for the webui while the rest of
clients are happily using anonymous principal. Another minus is that we
introduce another n principals (where N is number of masters) which
will (at least initially) be used only for FAST channel generation and
nothing else.

2.) use anonymous principal for FAST channel. The advantage is that when
PKINIT is configured properly (DL1), we use the same mechanism for
FAST/2FA in the whole topology regardless of the use case (although as
we already discussed we will still have to pass custom anchor location
during password auth in the framework). A disadvantage is that in
no-PKINIT case we would have to use certmonger's local CA to issue
PKINIT cert, fetch the local CA cert and store it somewhere in (e.g. in
/var/kerberos/kdc/cacert.pem). On DL-bump or if someone wishes to
enable PKINIT we have to re-issue the certs by IPA and replace the CA
cert by IPA-issued one (this can be made part of
ipa-server-certinstall).

I feel I can not really decide which approach is better. The first one is
consistent with regard to PKINIT status and domain level but inconsistent with
the client vs. server FAST channel generation mech. The second one uses common
mechanism everywhere, but in no-PKINIT case we must 'fake' the anonymous PKINIT
by specially issued certificates which feels a bit hacky to me.

Alexander Bokovoy

2017-03-24 08:53:49 UTC

Just to recap the reason to support PKINIT locally is for supporting 2FA
in the WebUI which is a hard requirement.

Correct.

The option is -X x509_anchor=...

I think we can do that in pre-start hook in httpd.service.d/ipa.conf

--
/ Alexander Bokovoy

Ok so let's forget for a moment about the clients and focus on framework and
1.) use a custom principal for FAST channel, e.g. ipaapi/`hostname`
This is analogous to how were things done before privilege
separation/PKINIT work. A plus is that you configure it once and it
works regardless of whether PKINIT is configured properly or not. A
minus is that when PKINIT is configured you are still using
one special mechanism for FAST/2FA for the webui while the rest of
clients are happily using anonymous principal. Another minus is that we
introduce another n principals (where N is number of masters) which
will (at least initially) be used only for FAST channel generation and
nothing else.
2.) use anonymous principal for FAST channel. The advantage is that when
PKINIT is configured properly (DL1), we use the same mechanism for
FAST/2FA in the whole topology regardless of the use case (although as
we already discussed we will still have to pass custom anchor location
during password auth in the framework). A disadvantage is that in
no-PKINIT case we would have to use certmonger's local CA to issue
PKINIT cert, fetch the local CA cert and store it somewhere in (e.g. in
/var/kerberos/kdc/cacert.pem). On DL-bump or if someone wishes to
enable PKINIT we have to re-issue the certs by IPA and replace the CA
cert by IPA-issued one (this can be made part of
ipa-server-certinstall).
I feel I can not really decide which approach is better. The first one is
consistent with regard to PKINIT status and domain level but inconsistent with
the client vs. server FAST channel generation mech. The second one uses common
mechanism everywhere, but in no-PKINIT case we must 'fake' the anonymous PKINIT
by specially issued certificates which feels a bit hacky to me.

Both Simo and me are for (2).

Note that we are not faking anything in the case of DL0. It is real
PKINIT with limited functionality only.

Martin Babinsky

2017-03-24 10:52:01 UTC

Just to recap the reason to support PKINIT locally is for supporting 2FA
in the WebUI which is a hard requirement.

Correct.

The option is -X x509_anchor=...

I think we can do that in pre-start hook in httpd.service.d/ipa.conf

--
/ Alexander Bokovoy

Both Simo and me are for (2).
Note that we are not faking anything in the case of DL0. It is real
PKINIT with limited functionality only.
--
/ Alexander Bokovoy

Right, since I myself am ambivalent on the issue I will update your PKINIT
design page (or you can update it) on Monday and implement the agreed procedure
for DL0/no-PKINIT handling next week. I have some code ready for the fixes in
upgrade/replication so Iihave already tampered with the affected codebase.

If somebody disagrees with the proposed solution he has today and Monday to
voice his concerns.

Simo Sorce

2017-03-24 16:10:18 UTC