Kubernetes Probes
Health checks are a simple way to let the system know if an instance of your app is working or not working. If an instance of your app is not working, then other services should not access it or send a request to it. Instead, requests should be sent to another instance of the app that is ready or retried at a later time. The system should also bring your app back to a healthy state.
By default, Kubernetes starts to send traffic to a pod when all the containers inside the pod start and restarts containers when they crash. While this may be “good enough” when starting with basic deployments, it is highly recommended to create custom health checks as per each application/container needs. Kubernetes offers two types of health checks — liveness probes and readiness probes
For each pod, the K8s application developer can periodically execute liveness and readiness probes:
Liveness Probe — determines whether the container is running or not, and under what circumstances is it appropriate to restart the pod? If the liveness probe fails, the kubelet kills the container, and the container is subjected to its restart policy.
Readiness Probe: determines whether the container is ready to service requests and under what circumstances should we take the pod out of the list of service endpoints.
A “Probe” is an executable action/diagnostic performed periodically by the kubelet on a container. There are three types of actions -
— Exec action — Executes a specified command in the container. The condition is considered successful if the command exits with zero as its exit code.
— TCPSocket action — Performs a TCP check against the container’s IP address on a specified port. The condition is considered successful if the port is open.
— HTTPGet action — Performs an HTTP GET request against the container’s IP address on a specified port and path. The condition is considered successful if the response has an HTTP status code greater than or equal to 200 and less than 400.
Each probe has one of three results:
— Success: The container passed the diagnostic.
— Failure: The container failed the diagnostic.
— Unknown: The diagnostic failed, so no action should be taken.
Readiness and liveness probes can be used in parallel for the same container. Using both can ensure that traffic does not reach a container that is not ready for it and that containers are restarted when they fail.
When should application developers use Liveness Probes?
— If the application starts up in a few seconds or less, then a liveness probe is probably unnecessary. If it takes more than a few seconds, it is recommended to put in a liveness probe to ensure that the container initializes without error rather than crashing.
— If the process in your container is able to crash on its own whenever it encounters an issue or becomes unhealthy, it is okay to not have a liveness probe. The kubelet will automatically perform the correct action in accordance with the Pod’s restartPolicy.
— To handle cases in which container is not responsive/hung and needs to be killed + restarted, a liveness probe is highly recommended along with the Pod’s ‘restartPolicy’ set to Always or OnFailure.
apiVersion: v1
kind: Pod
metadata:
name: sample-pod
spec:
containers:
- name: sample-container
image: k8s.gcr.io/busybox
ports:
- containerPort: 8080
args:
- /bin/sh
- -c
- touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
— — — — — — — — — — — — — — — -
livenessProbe:
exec:
command:
- cat
- /tmp/healthy
initialDelaySeconds: 5
periodSeconds: 5
— — — — — — — — — — — — — — — -
### To perform a probe, the kubelet sends an HTTP GET request to the server that is running in the container and listening on port 8080. If the handler for the server’s /healthz path returns a success code, the kubelet considers the container to be alive and healthy. If the handler returns a failure code, the kubelet kills the container and restarts it.
livenessProbe:
httpGet:
path: /healthz
port: 8080
httpHeaders:
- name: Custom-Header
value: Awesome
initialDelaySeconds: 3
periodSeconds: 3
— — — — — — — — — — — — — — — -
### the kubelet will attempt to open a socket to your container on the specified port. If it can establish a connection, the container is considered healthy, if it can’t it is considered a failure.
livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 15
periodSeconds: 20
— — — — — — — — — — — — — — — -When should application vendors use Readiness Probes?
In the containerized application is handling incoming network requests, a readiness probe is a must. Without a readiness probe, Kubernetes will immediately send network traffic to the application container after starting it, whether the application is ready or not. If the container starts dropping requests but hasn’t crashed, it will continue to receive traffic indefinitely.
apiVersion: v1
kind: Pod
metadata:
name: sample-pod
spec:
containers:
- name: sample-container
image: k8s.gcr.io/busybox
ports:
- containerPort: 8080
args:
- /bin/sh
- -c
- touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
readinessProbe:
exec:
command:
- cat
- /tmp/healthy
initialDelaySeconds: 5
periodSeconds: 5
— — — — — — — — — — — — — — — -
# Configuration for HTTP and TCP readiness probes is identical to liveness probes.
— — — — — — — — — — — — — — — -