Stop Putting Secrets in Environment Variables: Practical systemd-creds on Linux

Passwords in Environment= lines, API tokens in world-readable unit drop-ins, and EnvironmentFile=/etc/myapp.env that every process on the host can cat once it gets a shell — that pattern still shows up in homelab and production units alike.

systemd has a better primitive: credentials. They are limited-size binary or text blobs that the service manager loads at activation time, decrypts if needed, places in a private directory, and exposes to the service via $CREDENTIALS_DIRECTORY. The operator tool is systemd-creds.

This guide is operational. Commands and behavior come from systemd-creds(1), the Credentials section of systemd.exec(5), and the upstream System and Service Credentials document. Examples target systemd 250+ (encryption tooling); user-scoped encrypt/decrypt needs 256+. The host used for command verification here runs systemd 257.

Why not just use an env var?

Credentials fix the runtime side:

• Acquired at activation, released on deactivation, immutable while the service runs
• Files under $CREDENTIALS_DIRECTORY, mode-restricted to the service UID (and root)
• Optionally AES-256-GCM encrypted/authenticated at rest (TPM2 and/or host secret)
• Preferable to env for secrets; binary-safe; ~1 MB accumulated size limit per unit
• Work cleanly with RootDirectory= / RootImage= / portable services (host path not required inside the image)

Prerequisites

command -v systemd-creds
systemd-creds --version
# encrypt/decrypt/list/cat/setup: systemd 250+
# --user / --uid= encrypt: 256+

# Optional: see whether TPM2 is usable for credential protection
systemd-creds has-tpm2 || true

You need root (or equivalent) for host-key setup and for system-unit examples. Transient labs use systemd-run.

The five unit settings (and when to use each)

From systemd.exec(5):

Search paths for relative sources (also used by ImportCredential=):

• Plain: /etc/credstore/, /run/credstore/, /usr/lib/credstore/ (plus credentials already passed into the system)
• Encrypted: /etc/credstore.encrypted/, /run/credstore.encrypted/, /usr/lib/credstore.encrypted/
• Per-user manager: $XDG_CONFIG_HOME/credstore/, $XDG_RUNTIME_DIR/credstore/, $HOME/.local/lib/credstore/ (and .encrypted counterparts)

Inside the service:

$CREDENTIALS_DIRECTORY/<ID>   # primary interface
%d/<ID>                       # unit-file specifier for Environment=
${CREDENTIALS_DIRECTORY}/<ID> # ExecStart= interpolation
/run/credentials/<unit>       # system services only — prefer $CREDENTIALS_DIRECTORY

Lab 1 — Plain LoadCredential with a transient unit

Prove the plumbing before encryption:

# As root
echo -n 'hello-from-disk' > /root/demo-plain.txt
chmod 600 /root/demo-plain.txt

systemd-run -P --wait \
  -p LoadCredential=demo:/root/demo-plain.txt \
  systemd-creds cat demo
# → hello-from-disk

systemd-run -P --wait \
  -p LoadCredential=demo:/root/demo-plain.txt \
  systemd-creds list
# Shows name, size, and security state (secure/weak/insecure)

systemd-creds list and cat read $CREDENTIALS_DIRECTORY in the current execution context — that is why the lab runs them inside systemd-run with LoadCredential= attached.

Reference the path from a shell-style ExecStart without putting the secret in the environment:

systemd-run -P --wait \
  -p LoadCredential=demo:/root/demo-plain.txt \
  bash -c 'wc -c < "$CREDENTIALS_DIRECTORY/demo"; sha256sum "$CREDENTIALS_DIRECTORY/demo"'

Lab 2 — Encrypt with the host key

Default encrypt picks a key mode automatically: TPM2 when present and not in a container, host secret when /var/lib/systemd/ is persistent media, and both when both apply (host+tpm2). On VMs or hosts without a usable TPM, force the host key:

# Ensures /var/lib/systemd/credential.secret exists (also done implicitly on encrypt)
systemd-creds setup

# Encrypt stdin → file. Output basename becomes the embedded credential name.
echo -n 's3cr3t-db-password' | systemd-creds encrypt --with-key=host - /etc/credstore.encrypted/db-password.cred
chmod 600 /etc/credstore.encrypted/db-password.cred
# Directory may need creating first:
# mkdir -p /etc/credstore.encrypted && chmod 700 /etc/credstore.encrypted

# Round-trip check (name must match the embedded name = filename by default)
systemd-creds decrypt /etc/credstore.encrypted/db-password.cred
# → s3cr3t-db-password

# Wipe any leftover plaintext you used during creation
shred -u /root/demo-plain.txt 2>/dev/null || true

Key modes from systemd-creds(1) (--with-key= / -H / -T):

Encryption algorithm: AES-256-GCM, keyed from a SHA-256 derivation of the selected secret(s). Ciphertext is Base64.

Initrd / early-boot note: if a credential must decrypt before /var is available, do not bind it to the host secret. Use --with-key=tpm2 or auto-initrd.

Lab 3 — LoadCredentialEncrypted in a real unit drop-in

# Example consumer: a oneshot that only prints that the secret length is correct
cat >/etc/systemd/system/cred-demo.service <<'EOF'
[Unit]
Description=Credential demo service

[Service]
Type=oneshot
# Relative path → searched under credstore.encrypted (and friends)
LoadCredentialEncrypted=db-password
ExecStart=/usr/bin/bash -c 'test -n "$CREDENTIALS_DIRECTORY"; install -m 0400 "$CREDENTIALS_DIRECTORY/db-password" /run/cred-demo-check; wc -c /run/cred-demo-check; rm -f /run/cred-demo-check'
# Optional hardening that also implies PrivateMounts= on many distros:
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
EOF

systemctl daemon-reload
systemctl start cred-demo.service
systemctl status cred-demo.service --no-pager
journalctl -u cred-demo.service -n 20 --no-pager

Explicit path form (same effect):

LoadCredentialEncrypted=db-password:/etc/credstore.encrypted/db-password.cred

Failed decrypt/authenticate for LoadCredentialEncrypted= / SetCredentialEncrypted= fails the unit. Failed decrypt for something only pulled via ImportCredential= is skipped with a warning (credential absent).

Lab 4 — Paste ciphertext into the unit with SetCredentialEncrypted

When you want the secret in the drop-in (no separate .cred file), generate a pasteable line:

systemd-ask-password -n | systemd-creds encrypt --with-key=host --name=mysql-password -p - -

-p / --pretty prints a SetCredentialEncrypted=mysql-password: \ block. Pipe it into a drop-in:

mkdir -p /etc/systemd/system/myapp.service.d
systemd-ask-password -n | (
  echo '[Service]'
  systemd-creds encrypt --with-key=host --name=mysql-password -p - -
) >/etc/systemd/system/myapp.service.d/50-password.conf
chmod 600 /etc/systemd/system/myapp.service.d/50-password.conf
systemctl daemon-reload
systemctl restart myapp.service

The unit file remains world-readable in the usual case; the ciphertext is not useful without the host secret and/or TPM. That is the point of SetCredentialEncrypted= versus plaintext SetCredential=.

Name embedding: the credential name is stored inside the encrypted blob. Renaming mysql-password.cred to other.cred makes decrypt fail unless you override with --name=. Empty --name= disables the check (rarely what you want).

Lab 5 — Wire apps that do not know $CREDENTIALS_DIRECTORY yet

Many daemons want a path on the CLI or a single env var pointing at a file — not an env var containing the secret itself.

[Service]
User=myapp
LoadCredentialEncrypted=api-token
# Path-only env (the secret stays in the file; children do not inherit the bytes as env)
Environment=API_TOKEN_FILE=%d/api-token
ExecStart=/usr/local/bin/myapp --token-file ${CREDENTIALS_DIRECTORY}/api-token

%d expands to the credentials directory in unit settings; ${CREDENTIALS_DIRECTORY} works in Exec*= command lines. Prefer these over copying secrets into Environment=SECRET=….

For software that insists on reading /run/credentials/myapp.service/api-token, that path works for system units, but $CREDENTIALS_DIRECTORY is the portable interface (also correct for systemd --user).

Lab 6 — Propagate system credentials with ImportCredential

Container managers, hypervisors, and cloud IMDS can pass credentials into PID 1. Enumerate them:

systemd-creds --system list
systemd-creds --system cat some-name

Examples of ingress (from the credentials documentation):

• systemd-nspawn --set-credential=name:value / --load-credential=
• QEMU SMBIOS type 11: io.systemd.credential:name=value or io.systemd.credential.binary:name=<base64>
• Kernel cmdline: systemd.set_credential= / systemd.set_credential_binary= (visible via /proc/cmdline — not for secrets)
• ESP credentials via systemd-stub
• /run/credentials/@initrd/ import on initrd → host transition

Consume in a service:

[Service]
ImportCredential=mycred
# or rename:
# ImportCredential=my.original.cred:my.renamed.cred
# or glob:
# ImportCredential=app.*
ExecStart=/usr/bin/systemd-creds cat mycred

Well-known system credentials (provisioning, not day-2 app secrets) include:

• passwd.hashed-password.<user>, passwd.plaintext-password.<user>, passwd.shell.<user> → systemd-sysusers
• firstboot.locale, firstboot.keymap, firstboot.timezone, … → systemd-firstboot
• tmpfiles.extra → tmpfiles
• vmm.notify_socket → PID 1 READY notification (e.g. vsock:CID:PORT)
• ssh.authorized_keys.root (IMDS import path)

See systemd.system-credentials(7) for the full inventory. Conditional start: ConditionCredential= / AssertCredential=.

User services (systemd 256+)

Encrypt for the per-user manager so the key material incorporates UID, username, and machine-id:

echo -n 'user-secret' | systemd-creds encrypt --user --uid=self - ~/credstore.encrypted/user-secret.cred
# Load from a user unit with LoadCredentialEncrypted=user-secret
# and store under ~/.config/credstore.encrypted/ or the paths listed above

Operational checklist

1. Create ciphertext with systemd-creds encrypt (--with-key=host in labs without TPM; default auto on real hardware).
2. Store under /etc/credstore.encrypted/ (mode 0700 dir, 0600 files) or embed with -p → SetCredentialEncrypted=.
3. Declare LoadCredentialEncrypted= / ImportCredential= on the unit; avoid plaintext Environment= for secrets.
4. Consume via $CREDENTIALS_DIRECTORY or %d path references.
5. Sandbox with ProtectSystem=, PrivateTmp=, ProtectHome=, or at least PrivateMounts= so other units cannot see the credentials mount.
6. Rotate by re-encrypting and restarting the unit; ciphertext is immutable for the life of an activation.
7. Backup /var/lib/systemd/credential.secret with the same care as disk-encryption keys if you rely on host binding — losing it bricks host-bound credentials.
8. Expiry (optional): systemd-creds encrypt --not-after=… embeds a hard fail-after timestamp checked at decrypt.

Priority when the same ID is configured multiple ways (from systemd.exec(5)): LoadCredential* / ImportCredential= win over SetCredential*. SetCredential= is a useful default that is overridden when a real credential appears.

Boundaries (what this article is not)

• Not SOPS + age in Git (repo encryption / GitOps). Credentials are the node-local runtime injection layer after deploy.
• Not Clevis + Tang NBDE or systemd-cryptenroll (volume unlock at boot).
• Not systemd-homed per-user home encryption.
• Not a full replacement for a corporate secret manager — but an excellent native fit for machine-local service secrets on systemd hosts.
• Not varlinkctl / D-Bus service APIs, linger/user managers, or image tooling from recent posts — only the credential object model and systemd-creds CLI.

Cleanup

systemctl stop cred-demo.service 2>/dev/null || true
rm -f /etc/systemd/system/cred-demo.service
rm -f /etc/systemd/system/myapp.service.d/50-password.conf
rm -f /etc/credstore.encrypted/db-password.cred
systemctl daemon-reload
# Keep /var/lib/systemd/credential.secret unless you intend to invalidate host-bound creds

References

• systemd-creds(1) — list, cat, setup, encrypt, decrypt, key modes, examples
• systemd.exec(5) — Credentials — LoadCredential*, SetCredential*, ImportCredential=, $CREDENTIALS_DIRECTORY, size limit
• System and Service Credentials — design goals, encryption, ingress paths, well-known credentials, search paths
• systemd.system-credentials(7) — well-known system credential names
• systemd-nspawn(1) — --set-credential= / --load-credential=

Stop putting long-lived secrets in Environment= and world-readable env files. Encrypt once with systemd-creds, load with LoadCredentialEncrypted= or SetCredentialEncrypted=, and let the service read a file from $CREDENTIALS_DIRECTORY like any other path-based secret API — with activation-time decrypt and per-unit isolation included.