As more organizations shift towards cloud native software approach, whether it be tight or loosely coupled technique, the challenging part is to provide developers with development environments that can support a smooth transition.
On the cloud native palette there are a couple of tools that can provide this loop for developers with one topic in focus - “write code”.
What is Skaffold?
Skaffold is an open-source project released by Google with some key areas that covers:
- It is lightweight client-side only tool, that developers can easily install on local machine
- It must works everywhere, meaning easy share project with other developers
- If covers all container and kubernetes features that makes developers productive more
- It must provide continuous feedback to developers while developing
Now that I get your attention, shall we dive a little further?
Prerequisites
In order to follow my presentation, I’ve prepared code available for you at skaffold-demo
There is Makefile, just in case that you’re wish to follow along with the reading this blog
➜ skaffold-demo git:(main) ✗ make Usage: make [ COMMAND ] Commands: all Run all targets tools Install necessary tools cluster Install local cluster ingress Setup ingress-nginx controller skaffold Generate voting-app images certs Make locally trusted development certificates hosts Add hosts to /etc/hosts
When we finish this setup you can use make all to get all-at-once prepared, but for now let’s start step-by-step.
By that said, use make tools and you will be prepared on tools that we will need on local machine
*in my case this is already up-to-date
➜ skaffold-demo git:(main) ✗ make tools 🍺 Install necessary tools brew install --quiet homebrew/core/kind brew install --quiet homebrew/core/kubernetes-cli brew install --quiet homebrew/core/helm brew install --quiet homebrew/core/skaffold brew install --quiet homebrew/core/mkcert brew install --quiet homebrew/core/nss
Setup local cluster
We will use Kind as a local cluster that operates over Docker installed on your machine.
I will assume that you know what Docker is and that it is already installed on your machine.
As a good practice I always run the below command to keep the machine clean.
docker system prune --all --force --volumes
Now run make cluster and wait couple of minutes (time for coffee)
➜ skaffold-demo git:(main) ✗ make cluster 🚀 Install local cluster kind create cluster --config /Users/igor/Documents/CyberLab/skaffold-demo/k8s/kind.yaml --wait 5m Creating cluster "develop" ... ✓ Ensuring node image (kindest/node:v1.32.2) 🧊 ✓ Preparing nodes 🧱 ✓ Writing configuration 📝 ✓ Starting control-plane 🧑✈️ ✓ Installing CNI 🌐 ✓ Installing StorageClass 💾 ✓ Joining worker nodes 🧩 ✓ Waiting ≤5m0s for control-plane = Ready 🏁 ✓ Ready after 1s 🎉 Set kubectl context to "kind-develop" You can now use your cluster with: kubectl cluster-info --context kind-develop Have a nice day! 👋
At this point you should get pointed to kind context locally as
➜ skaffold-demo git:(main) ✗ kubectl config current-context kind-develop ➜ skaffold-demo git:(main) ✗ kubectl cluster-info Kubernetes control plane is running at https://127.0.0.1:54784 CoreDNS is running at https://127.0.0.1:54784/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Setup ingress controller
You can skip this if you want, but I prefer to set up ingress and prepare later locally trusted development certificates. If want to follow use make ingress and wait until we met the condition meaning that’s all good and running to be used inside this local cluster.
➜ skaffold-demo git:(main) ✗ make ingress 🧩 Setup ingress-nginx controller kubectl apply --filename /Users/igor/Documents/CyberLab/skaffold-demo/k8s/ingress/deploy.yaml namespace/ingress-nginx created serviceaccount/ingress-nginx created role.rbac.authorization.k8s.io/ingress-nginx created role.rbac.authorization.k8s.io/ingress-nginx-admission created rolebinding.rbac.authorization.k8s.io/ingress-nginx created rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created clusterrole.rbac.authorization.k8s.io/ingress-nginx created clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created service/ingress-nginx-controller created service/ingress-nginx-controller-admission created deployment.apps/ingress-nginx-controller created job.batch/ingress-nginx-admission-create created job.batch/ingress-nginx-admission-patch created ingressclass.networking.k8s.io/nginx created validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created ⏳ Wait condition to met kubectl wait --namespace ingress-nginx --for=condition=ready pod \ --selector=app.kubernetes.io/component=controller \ --timeout=120s pod/ingress-nginx-controller-697950d654-2m7zc condition met
Provide registry
This section we need to provide in order to tag our local images. It will ask you for input based on you desired registry. In this case Dockerhub as you can see below
⬇️ Configure DockerHub registry Provide your DockerHub account: igor992 ⬇️ Updating configuration with registry: igor992 ✅ Registry configuration updated
NOTE: For this demo, as you will see later we will not push those images, instead we will use them locally from Kind cluster. If you want for any reason to push those to Dockerhub, you will need to prepare connection and change skaffold.yaml at section local.push = true
As placeholder code has <registry> on two location helm values and skaffold
When you provide above information those will be changed.
Generate images
Now that we have a cluster ready for usage, we need to prepare images locally for our demo application.
For this we use make skaffold
➜ skaffold-demo git:(main) ✗ make skaffold 🧱 Using skaffold to generate starting images cd voting-app && \ export IMAGE_TAG=latest && \ skaffold build --profile vote-images,--seed-image && \ skaffold build --profile vote-images,--seed-image Generating tags... - igor992/result -> igor992/result:latest - igor992/vote -> igor992/vote:latest - igor992/worker -> igor992/worker:latest Checking cache... - igor992/result: Not found. Building - igor992/vote: Not found. Building - igor992/worker: Not found. Building Starting build... Found [kind-develop] context, using local docker daemon. Building 3 artifacts in parallel Building [igor992/result]... Target platforms: [linux/arm64] #0 building with "default" instance using docker driver
As you can see it will use at this point something new as skaffold build –profile
This is coming from skaffold.yaml spec that we have under voting-app directory.
Here build act as control how artifact will be build and –profile control order of building those artifacts, as we have two profiles minus sign e.g. -seed-image excludes that profile to be executed.
Trust development cluster
We will now add these locally trusted certificates for our development application by make certs
➜ skaffold-demo git:(main) ✗ make certs 🔐 Make locally trusted development certificates cd .tls && \ mkcert "*.kind.local" && \ kubectl create namespace dev && \ kubectl --namespace dev create secret tls demo-cert \ --key=._wildcard.kind.local-key.pem \ --cert=._wildcard.kind.local.pem && \ mkcert --install Created a new certificate valid for the following names - "*.kind.local" 🔎 Reminder: X.509 wildcards only go one level deep, so this won’t match a.b.kind.local 🧠 ✅ The certificate is at "./_wildcard.kind.local.pem" and the key at "./_wildcard.kind.local-key.pem" 🗝️ 📅 It will expire on 1 June 2027 🎉 namespace/dev created secret/demo-cert created The local CA is already installed in the system trust store! 👍 The local CA is already installed in the Firefox trust store! 👍 The local CA is already installed in Java’s trust store! 👍
*I’ll not explain above part on this blog
Last one in order to use configured ingress with trusted certificates we need to change /etc/hosts over make hosts.
This will require from you to allow this (type your password) so we can add two entries as
127.0.0.1 dev-vote.kind.local dev-result.kind.local
Deploy application
*Application is for demo purpose only and not focused on best-precites on Docker images and other container topics.
Under the voting-app directory you will see skaffold.yaml definition. Let’s give a short introduction into this configuration before we use it.
In this demo we are going to use stable skaffold/v3 schema version followed by three profiles.
- local-develop
- vote-images
- seed-image
Profile vote-images and seed-image we already used over make skaffold step before.
This prepared locally for us three images for result, vote and worker services and one more seed-data, to be able to provide fake clicks and populate the database.
➜ voting-app git:(main) ✗ docker images -a REPOSITORY TAG IMAGE ID CREATED SIZE igor992/seed-data 4d58b8c 4d58b8c6cf34 18 minutes ago 132MB igor992/seed-data latest 4d58b8c6cf34 18 minutes ago 132MB igor992/worker 247d9c7 247d9c79b44f 18 minutes ago 139MB igor992/worker latest 247d9c79b44f 18 minutes ago 139MB igor992/result 7d6ef7e 7d6ef7e45c8e 18 minutes ago 138MB igor992/result latest 7d6ef7e45c8e 18 minutes ago 138MB igor992/vote 5706951 570695168d11 18 minutes ago 184MB igor992/vote latest 570695168d11 18 minutes ago 184MB kindest/node <none> 9ba0a97d7488 2 weeks ago 1.68GB
In order to deploy the whole stack on our local cluster, we are going to use a local-develop profile over command skaffold dev
➜ voting-app git:(main) ✗ skaffold dev Generating tags... - igor992/result -> igor992/result:8181ec5 - igor992/vote -> igor992/vote:8181ec5 - igor992/worker -> igor992/worker:8181ec5 Checking cache... - igor992/result: Found Locally - igor992/vote: Found Locally - igor992/worker:
As we already prepared images before, it will use local ones.
You can see part when we hit the deploy phase of our configuration defined under skaffold.yaml file (Starting deploy…)
Images will be loaded under a local cluster, in our case Kind.
To be able to check those use command below
docker exec -it $(kind get clusters | head -1)-worker crictl images
This will interact with one of the nodes locally and print images that’s there present.
➜ skaffold-demo git:(main) ✗ docker exec -it $(kind get clusters | head -1)-worker crictl images IMAGE TAG IMAGE ID SIZE docker.io/igor992/result 15439a086162c0cc9c8ebc01a2b789b730175785e5a2faf7eb550b51cc9407 138MB docker.io/igor992/vote 4d861b51985c372cbad13b79b9f9d8e24eb47e64bb172c947528b8009e9799 184MB docker.io/igor992/worker 1865a8b7f235469d1124e9105d934874bb1bef86793e 139MB docker.io/kindest/kindest v1.32.0@sha256:a8abac 1.68GB
*There are other images also present in node
Wait a bit to stabilize deployment as we have checks that perform if redis and database are first ready for connection.
Happy Helming! Waiting for deployments to stabilize... - dev:deployment/dev-worker is ready. [1/5 deployment(s) still pending] - dev:deployment/redis is ready. [2/5 deployment(s) still pending] - dev:deployment/dev-result is ready. [3/5 deployment(s) still pending] - dev:deployment/dev-vote is ready. [1/5 deployment(s) still pending] - dev:deployment/db is ready. Deployments stabilized in 1.144 second Port forwarding service/redis in namespace dev, remote port 6379 -> http://127.0.0.1:6379 Port forwarding service/db in namespace dev, remote port 5432 -> http://127.0.0.1:5432 Listing files to watch... - igor992/result - igor992/vote - igor992/worker Press Ctrl+C to exit Watching for changes...
At this point skaffold will perform next job for development lifecycle
- allow developers access to redis and database locally
- constantly loop over changes under three services
- if there is any code change made by developer it will sync those
Syncing 1 files for igor992/vote:4d861b51985c372cbad13b79b9f9d8e24eb47e64bb172c947528b8009e9799 Watching for changes... [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [18] [INFO] Worker reloading: /app/app.py modified [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [19] [INFO] Worker reloading: /app/app.py modified [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [20] [INFO] Worker reloading: /app/app.py modified [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [21] [INFO] Worker reloading: /app/app.py modified [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [22] [INFO] Worker exiting (pid: 19) [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [23] [INFO] Worker exiting (pid: 20) [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [24] [INFO] Worker exiting (pid: 21) [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [25] [INFO] Booting worker with pid: 24 [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [26] [INFO] Booting worker with pid: 25 [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:13 +0000] [27] [INFO] Booting worker with pid: 26 [dev-vote-5480b9c78b-655m5 dev-vote] [2025-08-02 14:36:14 +0000] [27] [INFO] Booting worker with pid: 27
In order for this to work, under the hood it uses the kubectl cp command and if you ever worked with this one, it is required to have tar binary installed under a running container. Another catch is to allow the service to reload its process on file changes. This will add hot code reloading ability so that we can live see changes after sync. For this we always have separate image that will skaffold use during development and not the same one as we bake for the production best-practice usages. In our case as we use gunicorn we added flag --reload to support this process changes.
Simulate voting
At this point as this dummy application supports one vote per time, we will simulate that we have “external” users to perform this voting for us.
For this we first needs to push image inside Kind over
kind load docker-image igor992/seed-data -n develop
where it will add this on each nodes for usage
➜ skaffold-demo git:(main) ✗ kind load docker-image igor992/seed-data:4d58b8c6cf34 --name develop Image: "igor992/seed-data:4d58b8c6cf34" with ID "sha256:fbf75323215720f3b3c3143c1dd73b1b09cba97f5959349614ecaf7d4a5ea0e7" not yet present on node "develop-control-plane", loading... Image: "igor992/seed-data:4d58b8c6cf34" with ID "sha256:fbf75323215720f3b3c3143c1dd73b1b09cba97f5959349614ecaf7d4a5ea0e7" not yet present on node "develop-worker", loading...
Then just use command
kubectl run seed-data --namespace dev --rm --tty -i --restart='Never' --image igor992/seed-data:latest
Wait a bit and it will populate “fake” votes on our application. That will be shown as result over *https://dev-result.kind.local
*And if you really want jump into locally database to validate those also
psql --host localhost --port 5432 --no-password --username postgres
SELECT id,vote FROM public.votes;
To prove that redis is also there use redis-cli INFO server or redis-cli ping to get response as pong
Conclusion
With this presentation I wanted to bring closer to developers a new approach on how development can be faster in terms of cloud computing service. If this implements another layer of shit-left practice, developers get feedback much faster even before the first real image gets deployed as permanent to be shared with the rest of the team.
With this said, keep learning each day something different and new is what makes a cloud native experience more fun.
For more information on Kubernetes and Skaffold, feel free to visit their official websites, which I also referenced while writing this blog.
Until my next blog, Happy Helming! 👋
