Managing Kubernetes often involves remembering a slew of kubectl commands and YAML syntax. What if you could interact with your cluster using natural language, right from your terminal, without sending potentially sensitive information to a third-party cloud service?
Enter kubectl-ai, a powerful kubectl plugin that integrates Large Language Models (LLMs) into your Kubernetes workflow. Combined with Ollama, which lets you run open-source LLMs like Google’s Gemma 3 locally, you can create a private, efficient, and conversational Kubernetes experience.
This guide will walk you through installing kubectl-ai and configuring it to use a locally running Gemma 3 model via Ollama.
Prerequisites:
kubectlinstalled and configured to access your Kubernetes cluster.- Basic familiarity with the command line/terminal.
- Sufficient hardware (RAM, CPU/GPU) to run the LLM model you choose. Check the Ollama model library for guidance.
Let’s get started!
Step 1: Download kubectl-ai
First, head over to the kubectl-ai releases page on GitHub and download the latest release suitable for your operating system and architecture (e.g., Linux amd64, Darwin arm64 for Apple Silicon Macs, etc.).
GoogleCloudPlatform kubectl-ai
Step 2: Install kubectl-ai
Once downloaded, you need to extract the binary and make it accessible in your system’s PATH. The following example assumes you downloaded the MacOS Darwin_arm64 version to your Downloads folder. Adjust the filename and paths accordingly for your system.
# Navigate to your Downloads directory (or wherever you saved the file)
cd Downloads
# Extract the archive (replace filename if needed)
tar -zxvf kubectl-ai_Darwin_arm64.tar.gz
# Navigate into the extracted directory
cd kubectl-ai_Darwin_arm64/
# Make the binary executable
chmod a+x kubectl-ai
# Move the binary to a location in your PATH (requires sudo/admin privileges)
# Common locations are /usr/local/bin or ~/bin if it's in your PATH
sudo mv kubectl-ai /usr/local/bin/
# Verify installation (optional)
kubectl-ai --version
Step 3: Install Ollama and Pull a Model
If you haven’t already, install Ollama from ollama.com. Ollama makes running various open-source LLMs on your local machine incredibly simple.
After installing Ollama, you need to pull a model. We’ll use Google’s gemma3 as an example. You can choose different sizes depending on your hardware capabilities (e.g., gemma3:4b for 4 billion parameters, gemma3:12b for 12 billion). Check the Ollama Gemma 3 library page for options and requirements.
Bash
# Pull the Gemma 3 model (e.g., the 4 billion parameter version)
ollama pull gemma3:4b
# You can list the models you have downloaded locally using following command
ollama list
(This will confirm gemma3:4b or your chosen model is available)
Step 4: Configure and Run kubectl-ai with Ollama
Now, let’s run kubectl-ai and tell it to use your local Ollama instance and the model you downloaded.
kubectl-ai --llm-provider ollama --model gemma3:4b --enable-tool-use-shim
--llm-provider ollama: Specifies Ollama as the backend.--model gemma3:4b: Tellskubectl-aiwhich Ollama model to use (make sure this matches a model fromollama list).--enable-tool-use-shim: This crucial flag allows the LLM to suggestkubectlcommands, whichkubectl-aican then execute for you .
Troubleshooting: macOS Security Warning
On macOS, you might encounter a security warning like: “Apple could not verify “kubectl-ai” is free of malware…”
This is macOS Gatekeeper preventing an unnotarized application downloaded from the internet from running. You can manually approve it:
- Find the exact path where
kubectl-ai, then remove the quarantine attribute using the path found above:
which kubectl-ai
xattr -d com.apple.quarantine /usr/local/bin/kubectl-ai
Now, try running the kubectl-ai --llm-provider... command from Step 4 again.
Step 5: Start Chatting!
If everything is set up correctly, running the command from Step 4 should present you with a prompt:
Hey there, what can I help you with today?
>>>
You can now ask questions about your Kubernetes cluster in natural language! For example:
>>> List all deployments in the 'production' namespace
>>> Show me the logs for the pod named 'web-server-xyz123'
>>> Generate a YAML for a simple Nginx deployment with 2 replicas
kubectl-ai will interpret your request and then execute it, showing you the results. You can also use it non-interactively using the -quiet flag, for example
kubectl-ai --llm-provider ollama --model gemma3:12b --enable-tool-use-shim -quiet "fetch logs for nginx app in hello namespace"
To learn more about this tool please visit the git hub repo here GoogleCloudPlatform kubectl-ai or run the following command in your terminal:
kubectl-ai -h
Conclusion
You’ve now successfully configured kubectl-ai to leverage a powerful LLM running entirely on your local machine via Ollama. This setup provides a conversational interface to Kubernetes, helping you query resources, troubleshoot issues, and even generate configurations, all while keeping your interactions private and potentially faster than relying on external cloud services.
Experiment with different questions and commands to see how kubectl-ai can streamline your Kubernetes workflow.
Akash Gupta
Senior VoIP Engineer and AI Enthusiast
AI and VoIP Blog
Thank you for visiting the Blog. Hit the subscribe button to receive the next post right in your inbox. If you find this article helpful don’t forget to share your feedback in the comments and hit the like/clap button. This will helps in knowing what topics resonate with you, allowing me to create more that keeps you informed.
Thank you for reading, and stay tuned for more insights and guides!
Leave a Reply