Debug a Kubernetes service locally

Debugging a service locally with Telepresence

Imagine you have a service running in a staging cluster, and someone reports a bug against it.

In order to figure out the problem you want to run the service locally... but the service depends on other services in the cluster, and perhaps on cloud resources like a database.

In this tutorial you'll see how Telepresence allows you to debug your service locally.

We'll use the telepresence command line tool to swap out the version running in the staging cluster for a debug version under your control running on your local machine.

Telepresence will then forward traffic from Kubernetes to the local process.

You should start a Deployment and publicly exposed Service like this:

copied
$ kubectl create deployment hello-world --image=datawire/hello-world
copied
$ kubectl expose deployment hello-world --type=LoadBalancer --port=8000

If your cluster is in the cloud you can find the address of the resulting Service like this:

Terminal

copied
$ kubectl get service hello-world
NAME          CLUSTER-IP     EXTERNAL-IP       PORT(S)          AGE
hello-world   10.3.242.226   104.197.103.123   8000:30022/TCP   5d

If you see <pending> under EXTERNAL-IP wait a few seconds and try again. In this case the Service is exposed at http://104.197.103.123:8000/.

On minikube you should instead do this to find the URL:

Once you know the address you can store its value (don't forget to replace this with the real address!):

And you send it a query and it will be served by the code running in your cluster:

Swapping your deployment with Telepresence

Important Starting telepresence the first time may take a little while, since $Kubernetes needs to download the server-side image.

At this point you want to switch to developing the service locally, replace the version running on your cluster with a custom version running on your laptop.

To simplify the example we'll just use a simple HTTP server that will run locally on your laptop:

We want to expose this local process so that it gets traffic from $Kubernetes, replacing the existing \`hello-world\` deployment.

Important: you're about to expose a web server on your laptop to the Internet.

This is pretty cool, but also pretty dangerous!

Make sure there are no files in the current directory that you don't want shared with the whole world.

Here's how you should run telepresence (you should make sure you're still in the /tmp/telepresence-test directory you created above):

This does three things:

* Starts a VPN-like process that sends queries to the appropriate DNS and IP ranges to the cluster.

* --swap-deployment tells Telepresence to replace the existing hello-world pod with one running the Telepresence proxy. On exit, the old pod will be restored.

* --run tells Telepresence to run the local web server and hook it up to the networking proxy.

As long as you leave the HTTP server running inside telepresence it will be accessible from inside the $Kubernetes cluster.

You've gone from this...

...to this:

We can now send queries via the public address of the \`Service\` we created, and they'll hit the web server running on your laptop instead of the original code that was running there before.

Wait a few seconds for the Telepresence proxy to startup; you can check its status by doing:

Once you see that the new pod is in \`Running\` state you can use the new proxy to connect to the web server on your laptop:

Finally, let's kill Telepresence locally so you don't have to worry about other people accessing your local web server by bringing it to the foreground and hitting Ctrl-C:

Now if we wait a few seconds the old code will be swapped back in.

Again, you can check status of swap back by running:

When the new pod is back to \`Running\` state you can see that everything is back to normal:

> What you've learned: Telepresence lets you replace an existing deployment with a proxy that reroutes traffic to a local process on your machine.

> This allows you to easily debug issues by running your code locally, while still giving your local process full access to your staging or testing cluster.

Now it's time to clean up the service:

Telepresence can do much more than this: see the reference section of the documentation, on the top-left, for details.

Still have questions? Ask in our Slack chatroom or file an issue on GitHub