secrets, credentials, and certificates in DevOps CI/CD pipelines
Secrets management in DevOps: credentials and certificates in CI/CD
11 min read
CI/CD needs secrets, yet sprawl and logs multiply risk. This guide covers a centralized pattern, Vault with GitLab, Kubernetes CSI mounts, and guardrails for rotation, access, and audit.
Why secrets break first in automated pipelines
Applications and platforms depend on database passwords, API keys, TLS material, SSH keys, and vendor credentials. In CI/CD that tension sharpens: jobs need programmatic access, while every copy in a repo, script, or log widens blast radius. Teams routinely fight secret sprawl across configs and env vars, accidental commits, painful manual rotation, over-broad access, and weak audit trails. Without a deliberate pattern, automation accelerates leakage instead of reliability.
Design for a central secrets plane
Anchor on a dedicated store such as HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault. Prefer just-in-time, short-lived credentials, scope each pipeline stage to the least privilege it needs, automate rotation, log every read, and isolate dev, staging, and production namespaces. The sketch below shows the high-level flow: the pipeline authenticates to the manager, receives constrained material, and only then touches downstream systems.
[CI/CD pipeline]
| authenticated request
v
[Secrets manager] <-> [Encrypted storage]
| short-lived token / dynamic secret
v
[Targets: apps & infra]Bootstrap Vault for pipelines (AppRole + KV v2)
This shell-oriented walkthrough enables AppRole for machines, defines a CI role with tight token and Secret ID TTLs, mounts a KV v2 engine at secret/, and seeds paths that match the GitLab jobs later in the article. Treat the sample values as placeholders—never reuse demo passwords in real environments.
# Enable AppRole for CI/CD
vault auth enable approle
vault write auth/approle/role/cicd-role \
secret_id_ttl=20m \
token_num_uses=10 \
token_ttl=1h \
token_max_ttl=2h \
secret_id_num_uses=40
vault secrets enable -path=secret kv-v2
vault kv put secret/dev/database username="dbadmin" password="supersecret"
vault kv put secret/staging/api-service api_key="staging-key"
vault kv put secret/prod/api-service api_key="prod-key-12345" cert="@/path/to/cert.pem"GitLab CI: authenticate once, read per job
Protected GitLab variables should hold the AppRole role and secret identifiers. The before_script exchanges them for a Vault token, then each job pulls only its path using jq against the KV v2 JSON envelope (.data.data.*). Staging and production stay separated; production stays manual behind human approval.
stages:
- test
- deploy
variables:
VAULT_ADDR: "https://vault.example.com:8200"
VAULT_ROLE_ID: "$CICD_ROLE_ID"
VAULT_SECRET_ID: "$CICD_SECRET_ID"
before_script:
- apt-get update && apt-get install -y jq curl
- |
VAULT_TOKEN=$(curl -s \
--request POST \
--data "{\"role_id\":\"$VAULT_ROLE_ID\",\"secret_id\":\"$VAULT_SECRET_ID\"}" \
"${VAULT_ADDR}/v1/auth/approle/login" | jq -r '.auth.client_token')
export VAULT_TOKEN
test:
stage: test
script:
- |
export DB_USERNAME=$(curl -s \
--header "X-Vault-Token: $VAULT_TOKEN" \
"${VAULT_ADDR}/v1/secret/data/dev/database" | jq -r '.data.data.username')
export DB_PASSWORD=$(curl -s \
--header "X-Vault-Token: $VAULT_TOKEN" \
"${VAULT_ADDR}/v1/secret/data/dev/database" | jq -r '.data.data.password')
echo "Running tests with database user: $DB_USERNAME"
./run-tests.sh
only:
- merge_requests
- branches
deploy_staging:
stage: deploy
script:
- |
export API_KEY=$(curl -s \
--header "X-Vault-Token: $VAULT_TOKEN" \
"${VAULT_ADDR}/v1/secret/data/staging/api-service" | jq -r '.data.data.api_key')
echo "Deploying to staging with API key: ${API_KEY:0:4}..."
./deploy-to-staging.sh
environment:
name: staging
url: https://staging.example.com
only:
- main
deploy_production:
stage: deploy
script:
- |
export API_KEY=$(curl -s \
--header "X-Vault-Token: $VAULT_TOKEN" \
"${VAULT_ADDR}/v1/secret/data/prod/api-service" | jq -r '.data.data.api_key')
echo "Deploying to production"
./deploy-to-production.sh
environment:
name: production
url: https://example.com
only:
- main
when: manualKubernetes: Vault CSI driver and SecretProviderClass
For cluster workloads, mount secrets through the Secrets Store CSI driver instead of baking files into images. SecretProviderClass maps Vault paths to keys; the pod mounts the CSI volume read-only under /mnt/secrets-store while optionally syncing material into a native Kubernetes Secret when secretObjects is set.
# SecretProviderClass for Vault (CSI)
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: app-secrets-vault
spec:
provider: vault
secretObjects:
- secretName: app-secrets
type: Opaque
data:
- objectName: db-username
key: username
- objectName: db-password
key: password
parameters:
vaultAddress: "https://vault.example.com:8200"
roleName: "k8s-role"
objects: |
- objectName: "db-username"
secretPath: "secret/data/prod/database"
secretKey: "username"
- objectName: "db-password"
secretPath: "secret/data/prod/database"
secretKey: "password"# Pod mounting the CSI volume
apiVersion: v1
kind: Pod
metadata:
name: app-pod
spec:
containers:
- name: app
image: myapp:latest
volumeMounts:
- name: secrets-store01-inline
mountPath: "/mnt/secrets-store"
readOnly: true
volumes:
- name: secrets-store01-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "app-secrets-vault"Operational guardrails: lifecycle, access, pipelines
Prefer dynamic credentials from databases or cloud IAM where available. Rotate on a schedule tied to risk, invalidate superseded values, and keep versions for rollback. Issue short-lived tokens, use mTLS between services when feasible, and lean on cloud managed identities (IRSA, Azure Managed Identity). In pipelines, ban secret values from logs, mask CI variables, scope secrets to the jobs that need them, run secret scanning on commits, and split infra-owned secrets from app-owned ones when duties must separate.
Evidence, compliance, and developer ergonomics
Log success and failure for every secret read, alert on anomalous geography or rate, review access quarterly, and export evidence for SOC 2, HIPAA, or PCI programs. Back up encrypted state with tested restore paths. Locally, use a Vault dev server or Compose stack, inject at runtime with envsubst or similar, reach for feature flags when you are toggling behavior—not smuggling long-lived secrets—and document each service’s required paths plus rotation owners. Training turns policy into habit.
Start narrow, then widen coverage
Effective secrets management is a system, not a single vault install: central storage, least privilege, automation, and auditability together shrink exposure while keeping DevOps fast. Pilot with the highest-risk credentials—production databases and payment APIs—then extend the same contracts to every environment. Treat the program as continuous: revisit policies as threats and stacks evolve.
Secret hygiene lands hardest where delivery friction already hides, so align controls with the release pipeline bottleneck checklist.
Audit logs only matter when telemetry is trustworthy; pair vault access traces with this observability baseline for small platform teams.