In a previous article we did an introduction into what Hashicorp Vault is and what it can be used for. This article is a continuation of that introduction. It is a practical guide on how to deploy Hashicorp Vault and use it for managing secrets effectively for the workloads running in the Kubernetes cluster.
Installing Hashicorp Vault
HashiCorp Vault can be installed using several methods, depending on your environment and use case. Below are the most common installation methods, explained step by step:
1. Install Vault Using a Precompiled Binary
This method is suitable for local development or testing.
Steps:
a. Download the Binary:
– Visit the HashiCorp Vault releases page.
– Download the appropriate binary for your operating system (e.g., Linux, macOS, Windows).
b. Extract the Binary:
– Unzip the downloaded file.
– Move the `vault` binary to a directory in your system’s `PATH` (e.g., `/usr/local/bin` on Linux/macOS).
Example for Linux/macOS:
unzip vault_<version>_<os>_<arch>.zip sudo mv vault /usr/local/bin/
c. Verify the Installation:
– Check that Vault is installed correctly:
vault --version
d. Start Vault in Development Mode:
– For testing purposes, you can start Vault in development mode:
vault server -dev
– This starts a Vault server with an in-memory storage backend and automatically initializes and unseals it.
2. Install Vault Using a Package Manager
This method is suitable for Linux systems using package managers like `apt` (Debian/Ubuntu) or `yum` (CentOS/RHEL).
Steps:
a. Add HashiCorp’s Repository:
– For Debian/Ubuntu:
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add - sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main" sudo apt update
– For CentOS/RHEL:
sudo yum install -y yum-utils sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
b. Install Vault:
– For Debian/Ubuntu:
sudo apt install vault
– For CentOS/RHEL:
sudo yum install vault
c. Verify the Installation:
– Check that Vault is installed correctly:
vault --version
d. Start Vault:
– Follow the official documentation to configure and start Vault in production mode.
3. Install Vault Using Docker
This method is suitable for containerized environments.
Steps:
a. Pull the Vault Docker Image:
– Pull the official Vault image from Docker Hub:
docker pull hashicorp/vault
b. Run Vault in Development Mode:
– Start a Vault container in development mode:
docker run -d --name=vault -p 8200:8200 hashicorp/vault server -dev
– This exposes Vault on port `8200`.
c. Access Vault:
– Open a browser or use `curl` to access Vault at `http://localhost:8200`.
d. Run Vault in Production Mode:
– For production, configure a `vault.hcl` file and mount it into the container:
docker run -d --name=vault -p 8200:8200 -v /path/to/vault.hcl:/vault/config/vault.hcl hashicorp/vault server -config=/vault/config/vault.hcl
4. Install Vault on Kubernetes Using Helm
This method is suitable for Kubernetes environments.
Steps:
a. Add the HashiCorp Helm Repository:
helm repo add hashicorp https://helm.releases.hashicorp.com helm repo update
b. Install Vault:
– Install Vault in a Kubernetes namespace (e.g., `vault`):
helm install vault hashicorp/vault --namespace vault --create-namespace
c. Initialize and Unseal Vault:
– Follow the Helm chart’s output to initialize and unseal Vault.
d. Access Vault:
– Use port-forwarding to access Vault:
kubectl port-forward svc/vault -n vault 8200:8200
– Access Vault at `http://localhost:8200`.
5. Install Vault Using Terraform
This method is suitable for infrastructure-as-code (IaC) environments.
Steps:
a. Write a Terraform Configuration:
– Use the `hashicorp/vault` provider to deploy Vault on a cloud platform (e.g., AWS, Azure, GCP).
Example for AWS:
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "vault" {
ami = "ami-0c02fb55956c7d316"
instance_type = "t2.micro"
key_name = "your-key-pair"
tags = {
Name = "vault-server"
}
}
resource "null_resource" "install_vault" {
provisioner "remote-exec" {
inline = [
"sudo apt update",
"sudo apt install -y unzip",
"curl -O https://releases.hashicorp.com/vault/<version>/vault_<version>_linux_amd64.zip",
"unzip vault_<version>_linux_amd64.zip",
"sudo mv vault /usr/local/bin/",
]
}
}
b. Apply the Terraform Configuration:
terraform init terraform apply
c. Configure and Start Vault:
– SSH into the instance and configure Vault using a `vault.hcl` file.
Vault is installed and running smoothly. Since our focus is on using Vault for Kubernetes, let us install a feature in the Kubernetes cluster that pull secrets from Vault and injects it into the pod via Secrets and Environment Variables.
What is ExternalSecrets
ExternalSecrets is a Kubernetes operator that simplifies the process of managing and synchronizing secrets from external secret management systems (like AWS Secrets Manager, HashiCorp Vault, Azure Key Vault, or Google Secret Manager) into Kubernetes Secrets. Think of it as a bridge between your Kubernetes cluster and external secret stores, allowing you to securely fetch and store sensitive information (like API keys, passwords, or certificates) without manually copying them into Kubernetes. You define an `ExternalSecret` resource that specifies which external secret to fetch and how to map it into a Kubernetes Secret, and the operator automatically handles the synchronization, ensuring your applications always have access to the latest secrets. This approach enhances security by centralizing secret management, reducing the risk of human error, and making it easier to rotate or update secrets across your infrastructure. ExternalSecrets is particularly useful in cloud-native environments where secrets are often stored in external systems and need to be securely injected into Kubernetes workloads.
Install ExternalSecrets in Kubernetes
To install the ExternalSecrets operator in a Kubernetes cluster using Helm, follow these steps:
Prerequisites:
1. A running Kubernetes cluster.
2. `kubectl` installed and configured to access the cluster.
3. `helm` (v3 or later) installed on your local machine.
Steps to Install ExternalSecrets via Helm:
1. Add the ExternalSecrets Helm Repository:
Add the official ExternalSecrets Helm repository to your local Helm client.
helm repo add external-secrets https://charts.external-secrets.io helm repo update
2. Create a Namespace for ExternalSecrets:
Create a dedicated namespace for the ExternalSecrets operator.
kubectl create namespace external-secrets
3. Install the ExternalSecrets Operator:
Use Helm to install the ExternalSecrets operator in the `external-secrets` namespace.
helm install external-secrets \ external-secrets/external-secrets \ -n external-secrets \ --wait
– `external-secrets/external-secrets`: Refers to the Helm chart in the repository.
– `-n external-secrets`: Specifies the namespace where the operator will be installed.
– `–wait`: Ensures the installation waits for the resources to be ready.
4. Verify the Installation:
Check that the ExternalSecrets operator pods are running.
kubectl get pods -n external-secrets
You should see pods with names like `external-secrets-<pod-id>` in a `Running` state.
Configure ExternalSecrets for Vault
To configure access to a HashiCorp Vault secret store using the ExternalSecrets operator, you need to:
1. Set up a Kubernetes Secret containing the Vault token or other authentication credentials.
2. Create a `SecretStore` or `ClusterSecretStore` resource to define the connection to Vault.
3. Define an `ExternalSecret` resource to fetch secrets from Vault and store them in a Kubernetes Secret.
Below is a step-by-step example:
Prerequisites:
– A running HashiCorp Vault instance.
– The ExternalSecrets operator installed in your Kubernetes cluster (see previous instructions for installation via Helm).
– A Vault token or other authentication method (e.g., AppRole, Kubernetes Auth) to access Vault.
Step 1: Create a Kubernetes Secret for Vault Authentication
If you’re using a Vault token for authentication, create a Kubernetes Secret to store the token:
apiVersion: v1 kind: Secret metadata: name: vault-token namespace: external-secrets # Replace with your namespace type: Opaque data: token: <base64-encoded-vault-token> # Encode your Vault token in base64
To encode your Vault token in base64:
echo -n "your-vault-token" | base64
Apply the Secret:
kubectl apply -f vault-token-secret.yaml
Step 2: Create a SecretStore for Vault
Define a `SecretStore` resource to configure the connection to HashiCorp Vault. Replace placeholders with your Vault server details.
apiVersion: external-secrets.io/v1beta1 kind: SecretStore metadata: name: vault-secret-store namespace: external-secrets # Replace with your namespace spec: provider: vault: server: "https://vault.example.com" # Replace with your Vault server URL path: "secret" # Replace with the path where your secrets are stored version: "v2" # Use "v1" for KV v1 or "v2" for KV v2 auth: tokenSecretRef: name: vault-token # Reference the Kubernetes Secret created earlier key: token # Key in the Secret containing the Vault token
Apply the `SecretStore`:
kubectl apply -f vault-secret-store.yaml
Step 3: Create an ExternalSecret to Fetch Secrets from Vault
Define an `ExternalSecret` resource to fetch a secret from Vault and store it in a Kubernetes Secret.
Example: Fetch a secret named `my-secret` from Vault and store it in a Kubernetes Secret named `my-kubernetes-secret`.
apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: my-external-secret namespace: default # Replace with your namespace spec: refreshInterval: 1h # How often to sync the secret secretStoreRef: name: vault-secret-store # Reference the SecretStore created earlier kind: SecretStore target: name: my-kubernetes-secret # Name of the Kubernetes Secret to create data: - secretKey: my-secret-key # Key in the Kubernetes Secret remoteRef: key: my-secret # Path to the secret in Vault property: my-key # Specific property in the Vault secret (optional)
Apply the `ExternalSecret`:
kubectl apply -f external-secret.yaml
Step 4: Verify the Kubernetes Secret
Check that the Kubernetes Secret has been created and populated with the secret from Vault:
kubectl get secret my-kubernetes-secret -n default -o yaml
The output should include the secret data fetched from Vault.
Alternative Authentication Methods
If you’re using a different authentication method (e.g., AppRole or Kubernetes Auth), the `SecretStore` configuration will vary. Here are examples:
AppRole Authentication:
apiVersion: external-secrets.io/v1beta1 kind: SecretStore metadata: name: vault-secret-store namespace: external-secrets spec: provider: vault: server: "https://vault.example.com" path: "secret" version: "v2" auth: appRole: path: "approle" # Path where AppRole auth is enabled in Vault roleId: "your-role-id" secretRef: name: vault-approle-secret # Kubernetes Secret containing secretId key: secretId
Kubernetes Auth:
apiVersion: external-secrets.io/v1beta1 kind: SecretStore metadata: name: vault-secret-store namespace: external-secrets spec: provider: vault: server: "https://vault.example.com" path: "secret" version: "v2" auth: kubernetes: role: "your-kubernetes-role" # Vault role for Kubernetes auth mountPath: "kubernetes" # Path where Kubernetes auth is enabled serviceAccountRef: name: default # Kubernetes ServiceAccount to use
Steps in Summary
1. Create a Kubernetes Secret to store Vault authentication credentials.
2. Define a `SecretStore` to configure the connection to Vault.
3. Create an `ExternalSecret` to fetch secrets from Vault and store them in Kubernetes Secrets.
4. Verify that the Kubernetes Secret has been created and populated.
Conclusion
Ensuring secrets and sensitive credentials are properly managed is crucial to the security of the Kubernetes cluster
