Managing applications in a Kubernetes cluster can be hectic sometimes. Multiple YAML manifests, Kustomize, or Helm Chart whatever option you decide to go for, you are still trapped in the “death of YAMLs”. They can become so cumbersome and eventually difficult to manage.
I have written about the Mono Chart, which can help reduce this, where a single Helm Chart can be used to deploy multiple services without duplicating YAML files.
While this is an amazing way to manage applications at scale, it is still composed of numerous YAML files that need to be maintained in repositories all the time.
There is another way to manage the life cycle of applications in Kubernetes that can reduce the number of YAML files that need to be managed. That option is the Kubernetes Operator.
Operators were originally built for internal Kubernetes management, but there is an open Integration to Kubernetes via Kubernetes Operators.
Resources such as Deployment, Service, Ingress, and Pod are built-in Kubernetes objects with internal controllers that handle their functionality and lifecycle.
This management happens from within the Control Manager with a concept called the “Control Loop”. This is the mechanism Kubernetes uses to manage the state of applications.
With Kubernetes Operators, this powerful feature is made available to everyone to build their own Custom Resource Definitions take advantage of the control loop feature of the Kubernetes control plane, and add further customization if needed.
What is a Kubernetes Operator
A Kubernetes Operator is a method of packaging, deploying, and managing a Kubernetes application. It extends the Kubernetes API to create, configure, and manage instances of complex applications on behalf of a Kubernetes user. Operators are built using custom resources and controllers, which allow them to automate tasks such as deployment, scaling, backups, and updates. By encapsulating operational knowledge into software, Operators enable applications to be more self-managing and resilient, reducing the need for manual intervention and making it easier to run stateful applications in a Kubernetes environment. Operators can be deployed with or without the OLM Framework(https://olm.operatorframework.io/).
Say you have an application to deploy, in Kubernetes, the bare minimum for an API in Kubernetes is a Deployment, Service, Ingress, and Secret. These are four resource definitions and in some cases four YAML files that need to be managed. With a Kubernetes Operator, you can define a CRD like APIService, and whenever this APIService is applied to the Kubernetes cluster, it automatically deploys an API service with the four resources mentioned previously. Apart from deploying them, it will manage the end-to-end life cycle (OLM Framework) of the application. When the application needs to be deleted, there will be no need to delete the four components, instead delete the APIService and this will automatically delete the four resources (Deployment, Service, Ingress, and Secret). A sample YAML file with the CRD could look like this:
apiVersion: ewere.io/apps/v1
kind: APIService
metadata:
name: BillingService
spec:
url: billing.ewere.tech
With the sample code above, an operator setup can have various CRDs for managing various types of applications such as; APIService, CronService, and MLService.
NB: The CRDs I used are for illustration purposes, any other name can be used for the resource definition.
Next let us look at the key components of a Kubernetes Operator.
Key Components:
- CRDs: Define the custom resources your Operator manages and the custom API
- Controller Deployment: Runs the Operator’s reconciliation logic, or the control loop mechanism
- RBAC: Ensures proper permissions are set to be able to communicate with the Kubernetes API via the RBAC Components such as (ClusterRole and ClusterRoleBinding)
CRD has been mentioned earlier, what does it do and how does it work in Kubernetes.
What is a CRD
A CustomResourceDefinition (CRD) in Kubernetes is an extension mechanism that allows users to define their custom resources and APIs, beyond the built-in resources like Pods, Services, and Deployments. CRDs enable the creation of new resource types tailored to specific applications or workflows, which can then be managed using Kubernetes’ declarative API and tools like kubectl. Once a CRD is defined, Kubernetes treats the custom resource like any native resource, allowing users to create, update, and delete instances of it. CRDs are foundational for building Operators and other advanced Kubernetes extensions, enabling the platform to adapt to a wide range of use cases and workloads. They are defined with the CustomResourceDefinition. The following is a sample of a CRD
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: myapps.example.com
spec:
group: ewere.io
versions:
– name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
replicas:
type: integer
image:
type: string
scope: Namespaced
names:
plural: myapps
singular: myapp
kind: MyApp
shortNames:
– ma
Popular Applications using k8s Operator
Kubernetes Operators are commonly used to deploy stateful applications. Such as databases. This is due to the peculiarity of managing them in Kubernetes. Operators orchestrates a standard deployment based on the specific requirements of a particular service. For instance, when deploying a MongoDB cluster in Kubernetes, the Operator ensures the cluster of 3 nodes (which translates to 3 pods) is properly configured with features such as replication, sharding and high availability which can be complex to setup without an operator. The following are list of other services that use operators for management:
- Prometheus on the kube-prometheus-stack
- Traefik Operator
- MariaDB Operator
- MongoDB Operator
To get a full list of existing operators, visit the OperatorHub registry (similar to Helm registry for Helm Chart) ArtifactHub.
Sample Setup
Setting up an Operator yourself requires some good understanding of the Kubernetes API, Control Loop mechanism and the CRD, which has been discussed previously.
The Operator SDK is an open-source tool for building, testing and packaging operators. An operator can either be built in Go, Ansible or Helm. I shall be using Helm in this example because it is suitable when you need to implement a Kubernetes Operator on an existing Helm Chart, no need to write new code like the Go and Ansible implementation. Building a Helm-based Operator streamlines application management on Kubernetes by leveraging existing Helm charts within the Operator Framework. This guide provides a concise walkthrough to set up a Helm-based Operator using the Operator SDK.
Prerequisites
Before you begin, ensure you have the following tools installed:
- Kubernetes Cluster: A running Kubernetes cluster (e.g., Minikube, Kind, or a cloud-based cluster).
- kubectl: The Kubernetes command-line tool.
- Helm: The package manager for Kubernetes.
- Operator SDK: The toolkit for building Kubernetes Operators.
Step 1: Install the Operator SDK
Download and Install:
Follow the official Operator SDK installation guide to install the SDK on your system.
Verify Installation:
Run the following command to ensure the SDK is installed correctly:
operator-sdk version
Step 2: Create a New Operator Project
Initialize the Operator:
Use the Operator SDK to create a new Helm-based Operator:
operator-sdk init –plugins=helm –domain=example.com –group=demo –version=v1alpha1 –kind=MyApp
Replace example.com, demo, v1alpha1, and MyApp with your desired domain, group, version, and kind.
Navigate to the Project Directory:
Move into the newly created project folder:
cd myapp-operator
Step 3: Add a Helm Chart
Create a Helm Chart:
Use the Operator SDK to scaffold a Helm chart for your application:
operator-sdk create api –group=demo –version=v1alpha1 –kind=MyApp –helm-chart=myapp-chart
Replace myapp-chart with the name of your Helm chart.
Customize the Chart:
Edit the generated Helm chart located in the helm-charts directory to define your application’s resources and configurations.
Step 4: Build and Deploy the Operator
Build the Operator Image:
Build the Docker image for your Operator:
make docker-build docker-push IMG=<your-registry>/<your-username>/myapp-operator:v1.0.0
Replace <your-registry> and <your-username> with your container registry details.
Deploy the Operator:
Deploy the Operator to your Kubernetes cluster:
make deploy IMG=<your-registry>/<your-username>/myapp-operator:v1.0.0
Step 5: Create a Custom Resource (CR)
Define a Custom Resource:
Create a YAML file (e.g., myapp-cr.yaml) to define your custom resource:
apiVersion: demo.example.com/v1alpha1
kind: MyApp
metadata:
name: myapp-sample
spec:
url: myapp.com
Apply the Custom Resource:
kubectl apply -f myapp-cr.yaml
Step 6: Verify the Operator
Check Operator Logs:
View the Operator logs to ensure it’s running correctly:
kubectl logs -f deployment/myapp-operator-controller-manager -n myapp-operator-system
Verify Application Deployment:
Ensure your application resources are deployed as expected:
kubectl get pods,svc,deployments
Step 7: Clean Up
Delete the Custom Resource:
Remove the custom resource:
kubectl delete -f myapp-cr.yaml
Uninstall the Operator:
Uninstall the Operator from your cluster:
make undeploy
This is the Official Guide for more details
Conclusion
There are various ways to handle Kubernetes deployment and application management. Operators have a much lower level of abstraction than others (Helm and Kustomize) and hold great promise for the future of application management in Kubernetes.
You can also read how to Package a Helm Chart for deployments
