How to install Vault agent

Install and configure HashiCorp Vault agent.

Install Consul application

Create consul cluster, configure encryption and access control lists.

Create and configure vault application.

Create a secret

$ vault login

Token (will be hidden): 
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                  Value
---                  -----
token                hvs.BKpRwvIYboA6C9QrY5V7OnxA
token_accessor       Ie2zOw7FxgyNhx4KJopUpDTo
token_duration       ∞
token_renewable      false
token_policies       ["root"]
identity_policies    []
policies             ["root"]

Enable key-value storage.

$ vault secrets enable -version=2 kv

Success! Enabled the kv secrets engine at: kv/

Create sample secret.

$ vault kv put kv/agent/secret username=joe password=secret

==== Secret Path ====
kv/data/agent/secret

======= Metadata =======
Key                Value
---                -----
created_time       2022-07-20T22:34:15.209828875Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

Create access policy

Create sample policy.

$ cat << EOF | tee  agent.policy
path "kv/data/agent/*" {capabilities = ["read"]}
EOF

path "kv/data/agent/*" {capabilities = ["read"]}

Format a policy file.

$ vault policy fmt agent.policy

Success! Formatted policy: agent.policy

Display properly formatted file.

$ cat agent.policy

path "kv/data/agent/*" {
  capabilities = ["read"]
}

Upload policy.

$ vault policy write agent agent.policy

Success! Uploaded policy: agent

Configure AppRole auth method

Enable approle auth.

$ vault auth enable approle

Success! Enabled approle auth method at: approle/

Define role using role created earlier.

$ vault write auth/approle/role/agent  policies="agent"

Success! Data written to: auth/approle/role/agent

Get role id.

$ vault read auth/approle/role/agent/role-id

Key        Value
---        -----
role_id    0201b61d-c602-bdf9-94e5-6ce7ecd736a7

Display role definition.

$ vault read auth/approle/role/agent

Key                        Value
---                        -----
bind_secret_id             true
local_secret_ids           false
policies                   [agent]
secret_id_bound_cidrs      <nil>
secret_id_num_uses         0
secret_id_ttl              0s
token_bound_cidrs          []
token_explicit_max_ttl     0s
token_max_ttl              0s
token_no_default_policy    false
token_num_uses             0
token_period               0s
token_policies             [agent]
token_ttl                  0s
token_type                 default

Create empty secret.

$ vault write -force auth/approle/role/agent/secret-id

Key                   Value
---                   -----
secret_id             af93c5e5-c143-6196-de96-36f9516da065
secret_id_accessor    f0c2baf7-75b1-6796-5660-127c62c7eced
secret_id_ttl         0s

Configure login endpoint using role id and secret id.

$ vault write auth/approle/login \
    role_id=0201b61d-c602-bdf9-94e5-6ce7ecd736a7 \
    secret_id=af93c5e5-c143-6196-de96-36f9516da065

Key                     Value
---                     -----
token                   hvs.CAESIE7sv6PZJJCIoFnb7a47Ahe7KMH32pnJKJev1DypQyb5Gh4KHGh2cy5FaHU5dTBPcG9YTGdndnRQeUxQOWF0Nmo
token_accessor          QYmDVCWZInYBU39CdYrV0wdr
token_duration          768h
token_renewable         true
token_policies          ["agent" "default"]
identity_policies       []
policies                ["agent" "default"]
token_meta_role_name    agent

Test AppRole auth

$ curl -XPOST -d '{"role_id":"0201b61d-c602-bdf9-94e5-6ce7ecd736a7 ","secret_id":"af93c5e5-c143-6196-de96-36f9516da065"}' https://172.16.148.3:8200/v1/auth/approle/login

{"request_id":"375cf92d-07de-8e09-aca9-8d3f46f6e585","lease_id":"","renewable":false,"lease_duration":0,"data":null,"wrap_info":null,"warnings":null,"auth":{"client_token":"hvs.CAESINx5-iP0A7_W4BPoZQL1PzrFxD889kg8_sSunukCF3epGh4KHGh2cy5VTWZUVnF4MGowb0c2c2JqT2JXczh2N3Y","accessor":"1b2ibQVKMTAv073VfLAqoxKK","policies":["agent","default"],"token_policies":["agent","default"],"metadata":{"role_name":"agent"},"lease_duration":2764800,"renewable":true,"entity_id":"420d17e2-f0d6-96a0-a0c2-ec7e693e3fe2","token_type":"service","orphan":true,"mfa_requirement":null,"num_uses":0}}

Install Vault application

curl and unzip should be already installed during Consul agent install process.

Visit downloads page and download linux binary.

$ curl --silent \
       --remote-name \
       --output-dir /tmp \
       https://releases.hashicorp.com/vault/1.11.0/vault_1.11.0_linux_amd64.zip

Move archive to persistent location.

$ sudo mv /tmp/vault_1.11.0_linux_amd64.zip /opt/

Inspect downloaded archive.

$ unzip -l /opt/vault_1.11.0_linux_amd64.zip

Archive:  /opt/vault_1.11.0_linux_amd64.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
199518230  2022-06-17 23:04   vault
---------                     -------
199518230                     1 file

Extract vault binary.

$ sudo unzip -d /usr/bin /opt/vault_1.11.0_linux_amd64.zip vault

Archive:  /opt/vault_1.11.0_linux_amd64.zip
  inflating: /usr/bin/vault

Inspect binary file permissions.

$ ls -l /usr/bin/vault

-rwxr-xr-x 1 root root 199518230 Jun 17 23:04 /usr/bin/vault

Create vault group.

$ sudo groupadd --gid 864 vault

Create vault user.

$ sudo useradd --uid 864 --gid 864 \
               --shell /bin/false \
               --home-dir /etc/vault.d --no-create-home \
               vault

Create configuration directory.

$ sudo -u vault install --directory --group vault --owner vault --mode 700 /etc/vault.d

Create app data directory.

$ sudo -u vault install --directory --group vault --owner vault --mode 700 /opt/vault

Create empty environment file.

$ sudo -u vault touch /etc/vault.d/vault.env

Create empty configuration.

$ sudo -u vault touch /etc/vault.d/vault.hcl

Prevent memory from being swapped to disk.

$ sudo setcap cap_ipc_lock=+ep /usr/bin/vault

Create systemd service - it is the official packaged one.

$ sudo tee /usr/lib/systemd/system/vault-agent.service << 'EOF'
[Unit]
Description="HashiCorp Vault Agent - A tool for managing secrets"
Documentation=https://www.vaultproject.io/docs/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/vault.d/vault.hcl
StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
Type=exec
EnvironmentFile=/etc/vault.d/vault.env
User=vault
Group=vault
ProtectSystem=full
ProtectHome=read-only
PrivateTmp=yes
PrivateDevices=yes
SecureBits=keep-caps
AmbientCapabilities=CAP_IPC_LOCK
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
NoNewPrivileges=yes
ExecStart=/usr/bin/vault agent -non-interactive -config=/etc/vault.d/vault.hcl
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGINT
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
LimitNOFILE=65536
LimitMEMLOCK=infinity

[Install]
WantedBy=multi-user.target
EOF

[Unit]
Description="HashiCorp Vault Agent - A tool for managing secrets"
Documentation=https://www.vaultproject.io/docs/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/vault.d/vault.hcl
StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
Type=exec
EnvironmentFile=/etc/vault.d/vault.env
User=vault
Group=vault
ProtectSystem=full
ProtectHome=read-only
PrivateTmp=yes
PrivateDevices=yes
SecureBits=keep-caps
AmbientCapabilities=CAP_IPC_LOCK
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
NoNewPrivileges=yes
ExecStart=/usr/bin/vault agent -non-interactive -config=/etc/vault.d/vault.hcl
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGINT
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
LimitNOFILE=65536
LimitMEMLOCK=infinity

[Install]
WantedBy=multi-user.target

Reload systemd configuration.

$ sudo systemctl --system daemon-reload

Inspect service status.

$ systemctl status vault-agent

○ vault-agent.service - "HashiCorp Vault Agent - A tool for managing secrets"
     Loaded: loaded (/lib/systemd/system/vault-agent.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: https://www.vaultproject.io/docs/

Configure vault agent

Store role id and secret id.

$ echo 0201b61d-c602-bdf9-94e5-6ce7ecd736a7 | sudo -u vault tee /etc/vault.d/role-id

$ sudo chmod 600 /etc/vault.d/role-id

$ echo af93c5e5-c143-6196-de96-36f9516da065 | sudo -u vault tee /etc/vault.d/secret-id

$ sudo chmod 600 /etc/vault.d/secret-id

Create secret template.

$ sudo tee /etc/vault.d/secret.ctmpl << EOF
{{ with secret "kv/agent/secret" -}}
username = "{{ .Data.data.username }}"
password = "{{ .Data.data.password }}"
{{- end }}
EOF

{{ with secret "kv/agent/secret" -}}
username = "{{ .Data.data.username }}"
password = "{{ .Data.data.password }}"
{{- end }}

Create agent configuration. Update systemd service to use directory instead of a single file.

$ sudo -u vault tee /etc/vault.d/vault.hcl << EOF
vault {
  address = "https://172.16.148.3:8200"
}

auto_auth {
  method "approle" {
    config = {
      role_id_file_path = "/etc/vault.d/role-id"
      secret_id_file_path = "/etc/vault.d/secret-id"  
      remove_secret_id_file_after_reading = false
    }
  }
}

pid_file = "/opt/vault/vault.pid"

template {
  source = "/etc/vault.d/secret.ctmpl"
  destination = "/opt/vault/secret"
}
EOF

Enable server service.

$ sudo systemctl enable --now vault-agent

Created symlink /etc/systemd/system/multi-user.target.wants/vault-agent.service → /lib/systemd/system/vault-agent.service.

Inspect service status.

$ systemctl status vault

● vault-agent.service - "HashiCorp Vault Agent - A tool for managing secrets"
     Loaded: loaded (/lib/systemd/system/vault-agent.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2022-07-23 20:26:35 UTC; 9min ago
       Docs: https://www.vaultproject.io/docs/
   Main PID: 8759 (vault)
      Tasks: 8 (limit: 2241)
     Memory: 18.5M
        CPU: 325ms
     CGroup: /system.slice/vault-agent.service
             └─8759 /usr/bin/vault agent -non-interactive -config=/etc/vault.d/vault.hcl

Jul 23 20:26:35 vault-agent vault[8759]: 2022-07-23T20:26:35.106Z [INFO]  cache: received request: method=GET path=/v1/sys/internal/ui/mounts/kv/agent/secret
Jul 23 20:26:35 vault-agent vault[8759]: 2022-07-23T20:26:35.107Z [INFO]  cache.apiproxy: forwarding request: method=GET path=/v1/sys/internal/ui/mounts/kv/agent/secret
Jul 23 20:26:35 vault-agent vault[8759]: 2022-07-23T20:26:35.110Z [INFO]  auth.handler: renewed auth token
Jul 23 20:26:35 vault-agent vault[8759]: 2022-07-23T20:26:35.110Z [INFO]  cache: received request: method=GET path=/v1/kv/data/agent/secret
Jul 23 20:26:35 vault-agent vault[8759]: 2022-07-23T20:26:35.110Z [INFO]  cache.apiproxy: forwarding request: method=GET path=/v1/kv/data/agent/secret
Jul 23 20:30:53 vault-agent vault[8759]: 2022-07-23T20:30:53.970Z [INFO]  cache: received request: method=GET path=/v1/kv/data/agent/secret
Jul 23 20:30:53 vault-agent vault[8759]: 2022-07-23T20:30:53.971Z [INFO]  cache.apiproxy: forwarding request: method=GET path=/v1/kv/data/agent/secret
Jul 23 20:35:20 vault-agent vault[8759]: 2022-07-23T20:35:20.635Z [INFO]  cache: received request: method=GET path=/v1/kv/data/agent/secret
Jul 23 20:35:20 vault-agent vault[8759]: 2022-07-23T20:35:20.635Z [INFO]  cache.apiproxy: forwarding request: method=GET path=/v1/kv/data/agent/secret
Jul 23 20:35:20 vault-agent vault[8759]: 2022-07-23T20:35:20.658Z [INFO] (runner) rendered "/etc/vault.d/secret.ctmpl" => "/opt/vault/secret

$ sudo -u vault  cat /opt/vault/secret

username = "joe"
password = "secret"

Secret will be updated every five minutes.

One-time action

You can execute vault agent once, get secrets, parse templates, create the target files, remove auth data and quit.

$ sudo -u vault tee /etc/vault.d/vault.hcl << EOF
vault {
  address = "https://172.16.148.3:8200"
}

exit_after_auth = true

auto_auth {
  method "approle" {
    config = {
      role_id_file_path = "/etc/vault.d/role-id"
      secret_id_file_path = "/etc/vault.d/secret-id"  
      remove_secret_id_file_after_reading = true
    }
  }
}

template {
  source = "/etc/vault.d/secret.ctmpl"
  destination = "/opt/vault/secret"
}
EOF

Inspect service status after it used this configuration.

$ sudo systemctl status vault-agent

○ vault-agent.service - "HashiCorp Vault Agent - A tool for managing secrets"
     Loaded: loaded (/lib/systemd/system/vault-agent.service; enabled; vendor preset: enabled)
     Active: inactive (dead) since Sat 2022-07-23 21:03:38 UTC; 3s ago
       Docs: https://www.vaultproject.io/docs/
    Process: 9430 ExecStart=/usr/bin/vault agent -non-interactive -config=/etc/vault.d/vault.hcl (code=exited, status=0/SUCCESS)
   Main PID: 9430 (code=exited, status=0/SUCCESS)
        CPU: 97ms

Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [WARN] (clients) disabling nomad SSL verification
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO] (runner) creating watcher
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.696Z [INFO] (runner) starting
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.704Z [INFO]  auth.handler: renewed auth token
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO] (runner) stopping
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO]  template.server: template server stopped
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO] (runner) received finish
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO]  auth.handler: shutdown triggered, stopping lifetime watcher
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO]  auth.handler: auth handler stopped
Jul 23 21:03:38 vault-agent systemd[1]: vault-agent.service: Deactivated successfully.

Inspect service logs.

$ sudo journalctl -u vault-agent

Jul 23 21:03:38 vault-agent systemd[1]: Starting "HashiCorp Vault Agent - A tool for managing secrets"...
Jul 23 21:03:38 vault-agent systemd[1]: Started "HashiCorp Vault Agent - A tool for managing secrets".
Jul 23 21:03:38 vault-agent vault[9430]: ==> Vault agent started! Log data will stream in below:
Jul 23 21:03:38 vault-agent vault[9430]: ==> Vault agent configuration:
Jul 23 21:03:38 vault-agent vault[9430]:                      Cgo: disabled
Jul 23 21:03:38 vault-agent vault[9430]:                Log Level: info
Jul 23 21:03:38 vault-agent vault[9430]:                  Version: Vault v1.11.0, built 2022-06-17T15:48:44Z
Jul 23 21:03:38 vault-agent vault[9430]:              Version Sha: ea296ccf58507b25051bc0597379c467046eb2f1
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.660Z [INFO]  auth.handler: starting auth handler
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.661Z [INFO]  auth.handler: authenticating
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.661Z [ERROR] auth.approle: error removing secret ID file after reading: error="remove /etc/vault.d/secret-id: read-only fi>
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.661Z [INFO]  template.server: starting template server
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.661Z [INFO] (runner) creating new runner (dry: false, once: false)
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.661Z [INFO]  sink.server: starting sink server
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.662Z [WARN] (clients) disabling vault SSL verification
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.662Z [WARN] (clients) disabling nomad SSL verification
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.662Z [INFO] (runner) creating watcher
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO]  auth.handler: authentication successful, sending token to sinks
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO]  auth.handler: starting renewal process
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO]  sink.server: sink server stopped
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO]  sinks finished, exiting
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO]  template.server: template server received new token
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO] (runner) stopping
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO] (runner) creating new runner (dry: false, once: false)
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.695Z [INFO] (runner) creating watcher
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.696Z [INFO] (runner) starting
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.704Z [INFO]  auth.handler: renewed auth token
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO] (runner) stopping
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO]  template.server: template server stopped
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO] (runner) received finish
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO]  auth.handler: shutdown triggered, stopping lifetime watcher
Jul 23 21:03:38 vault-agent vault[9430]: 2022-07-23T21:03:38.802Z [INFO]  auth.handler: auth handler stopped
Jul 23 21:03:38 vault-agent systemd[1]: vault-agent.service: Deactivated successfully.

Awesome!

Install Consul application#

Create a secret#

Create access policy#

Configure AppRole auth method#

Test AppRole auth#

Install Vault application#

Configure vault agent#

One-time action#