A pod is the smallest unit of deployment in a Kubernetes cluster. But a pod on its own can not stand; with a delete command, the pod will no longer exist. To ensure a pod is not easily deleted, and Deployment is created, which handles the availability of pods via a Replicaset.
So, a pod configured in a deployment will recover based on the operator control loop mechanism. Setting up a deployment that is both scalable and secure goes beyond just the basics.
This article will delve into the configuration process and compare a standard deployment with one that is optimized for scalability and security.
What is a Scalable Deployment
A scalable deployment handles load in the event of an increase in request in a service or in the event of processing a request and handles it efficiently with minimal service disruption. Most deployments done on Kubernetes do not take advantage of the scalability features that the Kubernetes Deployment resource has to offer. Most of the time, the configuration termed scalable is by adding multiple replicas to ensure the load is shared across multiple pods. While this can be helpful, it is not as efficient as other configurations such as a rolling deployment strategy, which can be helpful during a deployment to ensure each deployment is rolled out in succession based on the roll-out strategy that has been configured. Although a scalable deployment also relies on things like HoritzonalPodAutoscaler, Cluster Autoscaler and other configurations, this article focuses on the configurations that can be done on the Deployment level.
What is a Secured Deployment
A secured deployment configuration in Kubernetes involves implementing best practices to protect workloads from vulnerabilities, unauthorized access, and attacks. This includes using Role-Based Access Control (RBAC) to restrict permissions, enabling network policies to control traffic flow, enforcing pod security standards, using secrets management for sensitive data, implementing mutual TLS (mTLS) for secure service communication, and scanning container images for vulnerabilities. Additionally, securing the API server, applying resource limits, and enabling logging and monitoring with tools like Prometheus and Falco help enhance the security posture of a Kubernetes deployment.
A basic deployment.yaml File
apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: selector: matchLabels: app: nginxdemo replicas: 1 template: metadata: labels: app: nginxdemo spec: containers: - name: nginxdemo image: nginx:1.14.2 ports: - containerPort: 80
This deployment file above is the bare minimum of what can be done with the deployment resource in Kubernetes. This deployment defines the name of the workload, the selector config that allows the deployment to be mapped with other resources like a Service, HorizontalPodAutoScaler, PodDisruptionBudget. The replicas defined in this sample are three; this automatically means that 3 pods will be created in this deployment metadata.label defines the labels that will be assigned to all the pods within the deployment. Labels are crucial for identifying and organizing resources, as they enable efficient management, selection, and filtering of pods. The containers section defines the number of containers in the pod, the name, the image, and the ports exposed by the pod.
This configuration does not handle proper rolling deployment for multiple replicas, meaning that a new deployment with this configuration can lead to service downtime before new pods get scheduled. The container has no security of any sort, which means a malicious actor can gain root access to the container.
Now let us see a better Deployment configuration
A Scalable and Secured deployment.yaml File
apiVersion: apps/v1
kind: Deployment
metadata:
name: "nginxapp"
spec:
selector:
matchLabels:
app: "nginxapp"
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
template:
metadata:
labels:
app: "nginxapp"
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: "nginxapp"
topologyKey: "kubernetes.io/hostname"
terminationGracePeriodSeconds: 70
containers:
- name: "nginxapp"
image: nginx:1.14.2
securityContext:
allowPrivilegeEscalation: false
runAsNonRoot: true
runAsUser: 1000
resources:
requests:
memory: 400Mi
cpu: "50m"
limits:
memory: 400Mi
ports:
- name: app-port
containerPort: 80
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 50"]
livenessProbe:
httpGet:
port: app-port
path: /
periodSeconds: 60
initialDelaySeconds: 10
readinessProbe:
httpGet:
port: app-port
path: /
periodSeconds: 60
initialDelaySeconds: 10
Breaking it down the Deployment file
apiVersion: apps/v1 kind: Deployment metadata: name: "nginxapp"
defines a resource named “nginxapp” under the apps/v1 API version. A Deployment is a higher-level abstraction that manages ReplicaSets to ensure the desired number of nginxapp pods run continuously. Although the YAML snippet is incomplete, a typical Deployment would specify a spec section containing the desired number of replicas, a pod template with a container definition (such as an Nginx image), and other configurations like environment variables, resource limits, and labels. Deployments enable rolling updates, rollback strategies, and self-healing by automatically replacing failed or unhealthy pods.
spec: selector: matchLabels: app: "nginxapp" replicas: 1 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 1
The selector.matchLabels field ensures that the Deployment controls only the Pods labeled app: “nginxapp”. The replicas: 1 setting specifies that a single instance of the Pod should be running at all times. The strategy section configures the update mechanism, using the RollingUpdate strategy, which allows updating Pods gradually to minimize downtime. The rollingUpdate parameters define how many Pods can be added or removed during an update: maxSurge: 1 permits one extra Pod to be created beyond the desired count, ensuring availability during updates, while maxUnavailable: 1 allows one Pod to be unavailable at a time, controlling disruption during the rollout process. This strategy ensures smooth and reliable updates with minimal service interruption.
spec: template: metadata: labels: app: "nginxapp" spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchLabels: app: "nginxapp" topologyKey: "kubernetes.io/hostname" terminationGracePeriodSeconds: 70
The template.metadata.labels section ensures that the Pods created by this Deployment are labeled with app: “nginxapp”, which must match the selector in the Deployment for proper management. The affinity section defines a podAntiAffinity rule that prevents multiple nginxapp Pods from being scheduled on the same node. Specifically, the requiredDuringSchedulingIgnoredDuringExecution field enforces a strict rule that ensures Pods with the app: “nginxapp” label are scheduled on different nodes based on the topologyKey: “kubernetes.io/hostname”, which represents the node name. This improves fault tolerance and availability by spreading Pods across nodes. Additionally, terminationGracePeriodSeconds: 70 specifies that when a Pod is terminated, it will have 70 seconds to shut down gracefully before being forcibly removed, allowing ongoing connections to close properly and ensuring a smooth application shutdown.
containers:
- name: "nginxapp"
image: nginx:1.14.2
securityContext:
allowPrivilegeEscalation: false
runAsNonRoot: true
runAsUser: 1000
resources:
requests:
memory: 400Mi
cpu: "50m"
limits:
memory: 400Mi
ports:
- name: app-port
containerPort: 80
This Kubernetes container specification defines a container named “nginxapp”, running the nginx:1.14.2 image with security hardening and resource constraints. The securityContext settings enhance security by disabling privilege escalation (allowPrivilegeEscalation: false), ensuring the container does not run as root (runAsNonRoot: true), and explicitly setting the user ID to 1000 (runAsUser: 1000). The resources section defines CPU and memory allocations, where the container requests400Mi of memory and 50m of CPU to guarantee minimal resource availability, while limiting memory usage to 400Mi to prevent excessive consumption. Additionally, the ports section exposes the application on port 80 (containerPort: 80), which is the default HTTP port for Nginx. This setup ensures a secure, efficient, and properly networked container deployment within a Kubernetes cluster. This configuration ensures efficient resource allocation and security best practices while running the Nginx application. In this configuration, the CPU limit is not configured deliberately, you can see more about that here.
containers: lifecycle: preStop: exec: command: ["/bin/sh", "-c", "sleep 50"] livenessProbe: httpGet: port: app-port path: / periodSeconds: 60 initialDelaySeconds: 10 readinessProbe: httpGet: port: app-port path: / periodSeconds: 60 initialDelaySeconds: 10
This Kubernetes container lifecycle and health check configuration ensures smooth application behavior during termination and monitoring. The lifecycle.preStop hook executes a command (/bin/sh -c “sleep 50”) when the container is about to stop, introducing a 50-second delay to allow in-flight requests to complete before termination. The livenessProbe checks if the application is running by sending an HTTP request to / on port app-port every 60 seconds, with an initial delay of 10 seconds to allow startup before the first check. Similarly, the readinessProbe ensures the application is ready to accept traffic by performing the same HTTP check every 60 seconds, also with an initial delay of 10 seconds. This configuration helps Kubernetes detect failures, prevent routing traffic to unready containers, and ensure smooth rolling updates. (Note: The indentation of periodSeconds and initialDelaySeconds in readinessProbe needs correction.)
From the comparison of both configurations we can see that the second is robust and has scalability and security configurations that ensure the resilience of workloads deployed in our Kubernetes cluster. This deployment file has been packaged in my Mono Helm Chart, you can see details about my Mono Helm Chart here.
Conclusion
It is essential to configure your Kubernetes workload according to a standard that will ensure the reliability, security, and scalability of applications running in the Kubernetes cluster. In this article, two aspects have been emphasized, which are scalability and security.
The topology spread architecture, which helps to spread pods across existing worker nodes, replicas which ensures multiple copies of pods are running implements the scalability of workloads.
While the securityContext ensures containers are protected from privilege escalation. All these ensure a scalable and secure workload is running in your Kubernetes cluster.
