How to access private GKE clusters from Google Cloud Build via the Identity Aware Proxy

In this blog post, I will show you how to access a private GKE cluster from Google Cloud Build via the Identity Aware Proxy. In the previous blog post, I explained how a simple HTTPS proxy allows you to connect to a private GKE cluster via IAP. In this blog I will explain the steps in a cloudbuild.yaml to create the following setup. deploy the IAP and GKE server proxy

To deploy the IAP and the GKE server proxy, follow the instructions in the README of github.com/binxio/simple-iap-proxy. After that, in your cloud build job, you:

• retrieve the required IAP parameters
• generate a certificate
• start the HTTPS proxy
• set the kubectl context to point to the proxy
  and you are ready to deploy!

retrieve the required IAP parameters

To connect via an IAP proxy, you need to have:

• the target URL at which the reverse proxy is listening
• the audience of the IAP application
• and the service account which grants access through the IAP proxy.
  This example stores these parameters as Google Secret Manager secrets. The following snippet makes these values available as environment variables in the cloud build job.

availableSecrets:
  secretManager:
    - env: AUDIENCE
      versionName: projects/${PROJECT_ID}/secrets/simple-iap-proxy-audience/versions/latest
    - env: SERVICE_ACCOUNT
      versionName: projects/${PROJECT_ID}/secrets/simple-iap-proxy-service-account/versions/latest
    - env: TARGET_URL
      versionName: projects/${PROJECT_ID}/secrets/simple-iap-proxy-target-url/versions/latest

generate the certificate

The first step generates a certificate which the HTTPS proxy will use to create secure connections:

  - id: generate certificate
    name: gcr.io/binx-io-public/simple-iap-proxy:0.5.2
    args: [
      "generate-certificate",
      "--key-file", "server.key",
      "--certificate-file", "server.crt",
      "--dns-name", "simple-iap-proxy"
    ]

start the HTTPS proxy

The next step starts the HTTPS proxy as background process in the cloud build job. As you can see, the Docker container is started on the cloudbuild network. This makes the proxy endpoint accessible to the other steps in the job.

  - id: start simple-iap-proxy
    name: gcr.io/cloud-builders/docker
    secretEnv:
      - AUDIENCE
      - TARGET_URL
      - SERVICE_ACCOUNT
    entrypoint: /bin/sh
    args:
      - -c
      - >
        docker run -d --rm -v /workspace:/workspace -w /workspace
        --network cloudbuild --name simple-iap-proxy
        gcr.io/binx-io-public/simple-iap-proxy:0.5.2 client
        --to-gke
        --port 8080
        --target-url $$TARGET_URL
        --iap-audience $$AUDIENCE
        --service-account $$SERVICE_ACCOUNT
        --use-default-credentials
        --key-file server.key
        --certificate-file server.crt &&
        sleep 2

set the kubectl context

The final step sets the kubectl context to point to your cluster via the proxy:

  - id: get credentials for cluster
    name: gcr.io/cloud-builders/gcloud
    entrypoint: gcloud
    args: [
        "container", "clusters",
        "get-credentials", "${_CLUSTER_NAME}",
        "--internal-ip",
        "--zone", "${_CLUSTER_ZONE}"
    ]

  - id: setup kubectl to use proxy
    name: gcr.io/cloud-builders/kubectl
    entrypoint: /bin/sh
    args:
      - -c
      - context_name=$(kubectl config current-context) &&
        kubectl config set clusters.$context_name.certificate-authority-data "$(base64 < server.crt)" &&
        kubectl config set clusters.$context_name.proxy-url https://simple-iap-proxy:8080 &&
        kubectl cluster-info

ready!

That is all there is to it. You can start using kubectl, helm or any other tool any way you want!

  - id: deploy to gke
    name: gcr.io/cloud-builders/kubectl
    entrypoint: kubectl
    args:
      - cluster-info

conclusion

Because Google Cloud Build allows you to start containers in the background, it is possible to start the simple-iap-proxy client to connect to a private GKE cluster via the Identity Aware Proxy.