Active Directory Authentication for Kubernetes Clusters

Active Directory Authentication for Kubernetes Clusters

January 21, 2020 1 By Eric Shanks

You’ve stood up your Kubernetes (k8s) cluster and are really looking forward to all of your coworkers deploying containers on it. How will you get everyone logged in? Creating local service accounts and distributing KUBECONFIG files (securely), seems like a real chore. This post will show how you can use Active Directory authentication for Kubernetes Clusters.

This post will use two projects, dex and gangway, to perform the authentication against ldap and return the Kubernetes login information to the user’s browser. The end result will look something like the screen below. The authenticated user will receive instructions on installing the client and setting up certificates for authentication.

Prerequisites

This post has several prerequisites that should be in place before setting up authentication with your Active Directory servers.

  1. A Working Kubernetes Cluster which connectivity to the AD infrastructure for Auth to take place.
  2. Cert-Manager should be installed, or be prepared to handle your own certificates for any new apps deployed. An article for using cert-manager can be found here. Cert-Manager will automatically deploy certificates to Dex/Gangway.
  3. An Ingress controller sending traffic to the apps that will be deployed in this post. Ingress Controller information can be found here.
  4. Permissions to make changes on the cluster.
  5. Create a shared secret that will be used in both gangway and dex configurations so that they may authenticate with each other. Use: “openssl rand -base64 32” and store this secret for use in this post.

Infrastructure Setup

We’ll need a couple of DNS names configured so that traffic will be delivered to dex and gangway from outside the cluster. Cert-Manager will need to be configured so that Dex and Gangway get their certificates installed on the Ingress Controller.

Additionally, if you’re looking for more information on how dex and gangway will interact with LDAP and the user’s browser, the section below will describe the authentication process.

Group Permissions Setup

For this lab, I want any users that are part of the “k8s_access” Active Directory group to have admin access to my cluster. First, create your Active Directory Group and place the users you wish to have access into this group. Then ensure you’ve got connection information for your AD servers handy, so we can use them in this first step.

We’ll also need a Kubernetes Role Binding so that when a user that is a member of this group authenticates, it will receive the proper permissions. Here is the role binding in my lab.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: Group
  name: k8s_access

Apply the role binding above to your cluster, or make your changes and apply.

Dex and Gangway

Now, assuming all of our prerequisites are in order, lets get to deploying our authentication tools into our Kubernetes cluster. As mentioned, we’ll use two tools, Dex and Gangway, to provide the authentication mechanisms for Active Directory.

Dex will serve as the identity provider that will validate our credentials with the Active Directory (ldap) identity store. Dex uses OpenID Connect to perform this validation.

Gangway will enable the end users to self-configure their kubectl configuration using the OpenID Connect Token provided by Dex after successful authentication. The full process can be seen in the below example.

  1. User attempts a login request to the gangway URL
  2. Gangway does a browser redirect to Dex through the user’s web browser
  3. Dex Responds to the request by responding with a login Form
  4. The user submits the login information to Dex
  5. Dex uses the login information to authenticate with LDAP to verify the credentials
  6. Dex provides a JWT back to gangway through a browser redirect
  7. Gangway provides the token needed to access the Kubernetes API via the web portal
  8. User takes the authentication information and places it in KUBECONFIG. Then is free to use KUBECONFIG to run commands against the Kubernetes API

For the deployment of Dex and Gangway, we’ll be building off the work of one of my colleagues, Alex Brand, who has a great tutorial of deploying Dex and Gangway in a Kubernetes cluster. We’ll only slightly modify it for use with Active Directory and the Cert-Manager issuers that we’ve used in a previous post. If you just want to learn Dex and Gangway, please check out his github project which is an excellent tutorial.

Deploy Dex

First we’ll deploy Dex, which is where our Active Directory Configuration will be necessary. Based on Alex’s tutorial, we’ll be deploying four items. A Configmap setup information for Dex as well as the ldap connector, and then the containers that are part of a k8s deployment. Then lastly we will deploy a service and ingress to provide access to this service from outside the cluster. First, let’s look at the configmap and then apply it.

The configmap below contains important information for Dex to do the authentication piece. There is connection information in here that is currently using non-secure ldap connections. The configuration also includes how dex should search ldap for your users, and then also list any groups those users are members of. The configmap that was used in my lab is shown below.

kind: ConfigMap
apiVersion: v1
metadata:
  name: dex
data:
  config.yaml: |
    issuer: https://dex.theithollowlab.com/dex
    storage:
      type: sqlite3
      config:
        file: dex.db

    # Configuration for the HTTP endpoints.
    web:
      http: 0.0.0.0:5556

    staticClients:
    - id: gangway
      redirectURIs:
      - https://gangway.theithollowlab.com/callback
      name: "Heptio Gangway"
      secret: mfgDcwBEgSgFehUFdQh2fhbftrgPOQWy0Q05gZgY8bs= #shared secret from prerequisites

    connectors:
    - type: ldap
      id: ldap
      name: LDAP
      config:
        host: 10.0.4.251:389  #Address of AD Server

        # Following field is required if the LDAP host is not using TLS (port 389).
        # Because this option inherently leaks passwords to anyone on the same network
        # as dex, THIS OPTION MAY BE REMOVED WITHOUT WARNING IN A FUTURE RELEASE.
        #
        insecureNoSSL: true

        # If a custom certificate isn't provide, this option can be used to turn on
        # TLS certificate checks. As noted, it is insecure and shouldn't be used outside
        # of explorative phases.
        #
        insecureSkipVerify: true

        # When connecting to the server, connect using the ldap:// protocol then issue
        # a StartTLS command. If unspecified, connections will use the ldaps:// protocol
        #
        # startTLS: true

        # Path to a trusted root certificate file. Default: use the host's root CA.
        # rootCA: /etc/dex/ldap.ca

        bindDN: CN=binduser,cn=users,dc=hollowaws,dc=local #user with access to search AD
        bindPW: Password123 #password of user with access to search AD

        # The attribute to display in the provided password prompt. 
        usernamePrompt: AD Username

        # User search maps a username and password entered by a user to a LDAP entry.
        userSearch:
          baseDN: dc=hollowaws,dc=local # BaseDN to start the search from. 
          # Optional filter to apply when searching the directory.
          filter: "(objectClass=person)"

          # username attribute used for comparing user entries. This will be translated
          # and combined with the other filter as "(<attr>=<username>)".
          username: sAMAccountName
          # The following three fields are direct mappings of attributes on the user entry.
          # String representation of the user.
          idAttr: sAMAccountName
          # Required. Attribute to map to Email.
          emailAttr: userPrincipalName
          # Maps to display name of users. No default value.
          nameAttr: displayName

        # Group search queries for groups given a user entry.
        groupSearch:
          # BaseDN to start the search from. It will translate to the query
          # "(&(objectClass=group)(member=<user uid>))".
          baseDN: OU=k8s,DC=hollowaws,DC=local
          # Optional filter to apply when searching the directory.
          filter: "(objectClass=group)"

          # Following two fields are used to match a user to a group. It adds an additional
          # requirement to the filter that an attribute in the group must match the user's
          # attribute value.
          userAttr: distinguishedName
          groupAttr: member

          # Represents group name.
          nameAttr: cnCode language: PHP (php)

Once the configmap has been configured for your environment and applied to your Kubernetes cluster, we can move on to deploying the rest of the dex components. Next, we’ll deploy our containers.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: dex
  name: dex
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dex
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: dex
    spec:
      containers:
      - command:
        - /usr/local/bin/dex
        - serve
        - /etc/dex/cfg/config.yaml
        image: quay.io/coreos/dex:v2.10.0
        imagePullPolicy: Always
        name: dex
        ports:
        - containerPort: 5556
          name: https
          protocol: TCP
        volumeMounts:
        - mountPath: /etc/dex/cfg
          name: config
      volumes:
      - configMap:
          items:
          - key: config.yaml
            path: config.yaml
          name: dex
        name: configCode language: JavaScript (javascript)

When the containers have been deployed through the above deployment manifest, a service and ingress rule should be deployed. For the ingress rule, be sure that you’ve updated your configuration to include the appropriate issuer deployed as part of the cert-manager prerequisites, and update your DNS names for the ingress rule for your environment.

---
kind: Service
apiVersion: v1
metadata:
  name: dex
spec:
  selector:
    app: dex
  ports:
  - port: 5556
    targetPort: https
    name: https
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: dex
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-production" #cert-manager issuer name
spec:
  tls:
  - hosts:
    - dex.theithollowlab.com
    secretName: dex-tls
  rules:
  - host: dex.theithollowlab.com #Your DNS Name for Dex
    http:
      paths:
      - backend:
          serviceName: dex
          servicePort: https
Code language: PHP (php)

Deploy Gangway

Now we’re ready to deploy Gangway which will be how the user interacts with the solution to get credentials. Gangway acts as the OIDC client.

Gangway will be deployed in its own namespace, and then a configmap with the gangway configs will be deployed first before our containers.

---
apiVersion: v1
kind: Namespace
metadata:
  name: gangway
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: gangway
  namespace: gangway
data:
  gangway.yaml: |
    # The cluster name. Used in UI and kubectl config instructions.
    # Env var: GANGWAY_CLUSTER_NAME
    clusterName: "hollowcluster"

    # OAuth2 URL to start authorization flow.
    # Env var: GANGWAY_AUTHORIZE_URL
    authorizeURL: "https://dex.theithollowlab.com/dex/auth" #replace the domain name with your domain

    # OAuth2 URL to obtain access tokens.
    # Env var: GANGWAY_TOKEN_URL
    tokenURL: "https://dex.theithollowlab.com/dex/token" #replace the domain name with your domain

    # Used to specify the scope of the requested Oauth authorization.
    scopes: ["openid", "profile", "email", "offline_access", "groups"]

    # Where to redirect back to. This should be a URL where gangway is reachable.
    # Typically this also needs to be registered as part of the oauth application
    # with the oAuth provider.
    # Env var: GANGWAY_REDIRECT_URL
    redirectURL: "https://gangway.theithollowlab.com/callback" #replace the domain name with your domain

    # API client ID as indicated by the identity provider
    # Env var: GANGWAY_CLIENT_ID
    clientID: "gangway"

    # API client secret as indicated by the identity provider
    # Env var: GANGWAY_CLIENT_SECRET
    clientSecret: "mfgDcwBEgSgFehUFdQh2fhbftrgPOQWy0Q05gZgY8bs=" #secret key from prerequisites again. This should match the Dex key

    # The JWT claim to use as the username. This is used in UI.
    # Default is "nickname".
    # Env var: GANGWAY_USERNAME_CLAIM
    usernameClaim: "sub"

    # The JWT claim to use as the email claim. This is used to name the
    # "user" part of the config. Default is "email".
    # Env var: GANGWAY_EMAIL_CLAIM
    emailClaim: "email"

    # The API server endpoint used to configure kubectl
    # Env var: GANGWAY_APISERVER_URL
    apiServerURL: https://cp.theithollowlab.com:6443 #This should be your k8s API URL

Just like we did with Dex, we’ll next deploy our containers which are part of the deployment manifest below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gangway
  namespace: gangway
  labels:
    app: gangway
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gangway
  template:
    metadata:
      labels:
        app: gangway
        revision: "1"
    spec:
      containers:
        - name: gangway
          image: gcr.io/heptio-images/gangway:v2.0.0
          imagePullPolicy: Always
          command: ["gangway", "-config", "/gangway/gangway.yaml"]
          env:
            - name: GANGWAY_SESSION_SECURITY_KEY
              valueFrom:
                secretKeyRef:
                  name: gangway-key
                  key: sesssionkey
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            limits:
              cpu: "200m"
              memory: "512Mi"
          volumeMounts:
            - name: gangway
              mountPath: /gangway/
          livenessProbe:
            httpGet:
              path: /
              port: 8080
            initialDelaySeconds: 20
            timeoutSeconds: 1
            periodSeconds: 60
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /
              port: 8080
            timeoutSeconds: 1
            periodSeconds: 10
            failureThreshold: 3
      volumes:
        - name: gangway
          configMap:
            name: gangway

And then lastly, we’ll deploy a service and ingress rule to allow communication to our gangway containers. Be sure to update the dns rules and issuers

---
kind: Service
apiVersion: v1
metadata:
  name: gangwaysvc
  namespace: gangway
  labels:
    app: gangway
spec:
  type: ClusterIP
  ports:
    - name: "http"
      protocol: TCP
      port: 80
      targetPort: "http"
  selector:
    app: gangway
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gangway
  namespace: gangway
  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-production" #your cert-manager issuer here
spec:
  tls:
  - secretName: gangway
    hosts:
    - gangway.theithollowlab.com #dns name previously configured for gangway
  rules:
  - host: gangway.theithollowlab.com #dns name previously configured for gangway
    http:
      paths:
      - backend:
          serviceName: gangwaysvc
          servicePort: http

Try it out!

Its that moment, you’ve been waiting for. Lets try to login with a user in our AD group that should have permissions to the cluster. Navigate to your gangway URL which in my case was https://gangway.theithollowlab.com.

Click the sign in button to continue. Then enter your AD Username and Password for the user in question. Notice that in the screenshot we were redirected to Dex for this step.

After logging in with the my test user, we’re presented with the option to grant access. This is a good sign. Click the “Grant Access” button.

Now, we’ll see that we’ve been redirected back to Gangway with the instructions on configuring kubectl for the command line.

After installing kubectl and executing the commands from the screen, you should be able to run a kubectl command against the cluster with no problem.

Here are those steps in my cli, and the last command is a simple get on the pods running in the default namespace.

Summary

Setting up a way to authenticate with a corporate directory for authentication is almost a must for most organizations. Its hard to have systems running everywhere with their own directory services so AD is pretty common. I hope this post helped show how you can connect your Kubernetes cluster to Active Directory to help ease this burden.

If you want more information around these projects, please check out these resources:

The code for this post can be found on this github repository for easier access: https://github.com/eshanks16/k8s-ldap

CategoryRecently Added
Tagscert-manager dex gangway k8s Kubernetes ldap oauth oidc

About The Author

Eric Shanks is a Principal Technical Marketing Manager at Portworx.

1 Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.