In an earlier article, I have explained that Cloud Run implements the Knative API. In this post I’ll show you how to use Cloud Run’s client libraries in Go to make API calls to Knative clusters (on Google or not) with code samples. (I’m guessing only like 10 people will ever need this, 9 of them probably at Google, but here we go).

Normally, the Cloud Run client library for Go (google.golang.org/api/run/v1) works with the serverless “Cloud Run fully managed” out of the box. However, getting it to work for “Cloud Run on GKE” or any other Knative cluster requires understanding of how to connect and authenticate to Kubernetes clusters.

This run/v1 package already has the type definitions like KService and RPCs like Create Service (and others) already ready to go. We just need to configure a proper HTTP client with authentication and TLS settings to support custom certs of the Kubernetes clusters.

Connecting to Knative API (on GKE)

You can authenticate to Kubernetes API on GKE/Anthos clusters using the Google client libraries (access_token authentication). So what we need to do in this case is:

  1. Get GKE cluster info (master CA certificate and Kubernetes API endpoint)
  2. Create a new Go net/http.Client with:
    • custom TLS config that authenticates the server over TLS using this CA certificate (this does not authenticate you as the caller, it is for verifying you’re talking to the Kubernetes API server and nobody is intercepting the traffic)
    • a http.Transport that adds your access_token to the outgoing requests
  3. Initialize and use a Cloud Run API client with this http.Client.

You can find the example code in this Gist or below:

package main

import (
	"context"
	"crypto/x509"
	"encoding/base64"
	"fmt"
	"net/http"

	"golang.org/x/oauth2"
	"golang.org/x/oauth2/google"
	"google.golang.org/api/container/v1"
	"google.golang.org/api/option"
	"google.golang.org/api/run/v1"
)

func main() {
	ctx := context.Background()
    caCert, masterIP, err := gkeClusterInfo(ctx, "project-id",
        "gke-cluster-name", "gke-cluster-zone")
	if err != nil {
		panic(err)
	}

	caCertPool := x509.NewCertPool()
	caCertPool.AppendCertsFromPEM([]byte(caCert))
	t := http.DefaultTransport.(*http.Transport).Clone()
	t.TLSClientConfig.RootCAs = caCertPool
	ts, err := google.DefaultTokenSource(ctx, "cloud-platform")
	if err != nil {
		panic(err)
	}
	tt := &oauth2.Transport{
		Base:   t,
		Source: ts}
	hc := &http.Client{Transport: tt}

	runService, err := run.NewService(ctx,
		option.WithHTTPClient(hc),
		option.WithEndpoint("https://"+masterIP))
	if err != nil {
		panic(err)
	}

	// List Services
	resp, err := runService.Namespaces.Services.List("namespaces/default").Do()
	if err != nil {
		panic(err)
	}
	fmt.Printf("%d kservices found\n", len(resp.Items))
}

func gkeClusterInfo(ctx context.Context, projectID, clusterName, zone string) ([]byte, string, error) {
	s, err := container.NewService(ctx)
	if err != nil {
		return nil, "", fmt.Errorf("failed to initialize gke api client: %w", err)
	}
	cluster, err := s.Projects.Zones.Clusters.Get(projectID, zone, clusterName).Do()
	if err != nil {
		return nil, "", fmt.Errorf("failed to get GKE cluster: %w", err)
	}

	cert, err := base64.StdEncoding.DecodeString(cluster.MasterAuth.ClusterCaCertificate)
	if err != nil {
		return nil, "", fmt.Errorf("error decoding cert: %v", err)
	}
	return cert, cluster.Endpoint, nil
}

Connecting to Knative API (using KUBECONFIG)

Just because a cluster is not running on Google Kubernetes Engine (GKE), it does not mean we cannot use the same Cloud Run client library. As long as you have a KUBECONFIG file, you can use this to connect to any Kubernetes cluster (GKE or elsewhere).

In this method, we will:

  1. use a KUBECONFIG file
  2. use Kubernetes client-go package parse the KUBECONFIG and give us an http.RoundTripper
  3. we will use this http.RoundTripper to configure an http.Client
  4. we will later use this http.Client to initialize a Cloud Run API client.

This http.RoundTripper we’ll get from client-go/rest.TransportFor method will be capable of detecting which authentication provider to use and how to authenticate, as well as TLS configuration.

Before you start:

  1. Make sure to import client-go auth plugins in your program like the following. This adds support for other clouds and OIDC providers. 
    import _ "k8s.io/client-go/plugin/pkg/client/auth"
  2. go-get the client-go package properly.
  3. Make sure you have KUBECONFIG environment variable set (or have a ~/.kube/config file)

then you can use the following snippet (GitHub Gist here) to initialize a Knative API client using Cloud Run client library:

package main

import (
	"context"
	"fmt"
	"net/http"

	"google.golang.org/api/option"
	"google.golang.org/api/run/v1"
	"k8s.io/client-go/rest"
	"k8s.io/client-go/tools/clientcmd"

	_ "k8s.io/client-go/plugin/pkg/client/auth"
)

func main() {
	kc := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
		clientcmd.NewDefaultClientConfigLoadingRules(),
		&clientcmd.ConfigOverrides{})
	kubeconfig, err := kc.ClientConfig()
	if err != nil {
		panic(err)
	}
	tr, err := rest.TransportFor(kubeconfig)
	if err != nil {
		panic(err)
	}
	hc := &http.Client{Transport: tr}
	ctx := context.Background()
	runService, err := run.NewService(ctx,
		option.WithHTTPClient(hc),
		option.WithEndpoint(kubeconfig.Host))
	if err != nil {
		panic(err)
	}

	// List Services
	resp, err := runService.Namespaces.Services.
		List("namespaces/default").Do()
	if err != nil {
		panic(err)
	}
	fmt.Printf("%d kservices found\n", len(resp.Items))
}

Hope this helps to the next person who spends an hour trying to figure out how to programmatically connect to Knative API (and maybe Cloud Run API) at the same time using the same client library.