Automatically add the identity file used by the SSH client to the OpenSSH authentication agent.

Inspect SSH client configuration.

$ cat ~/.ssh/config

Host pi-hole
  Hostname pi-hole.fishsilentcruise.space
  User milosz

Match user milosz
  IdentityFile ~/.ssh/milosz

List loaded identities.

$ ssh-add -l

The agent has no identities.

Perform SSH connection.

$ ssh pi-hole date

Enter passphrase for key '/home/milosz/.ssh/milosz': **********
Fri Oct 22 20:33:24 BST 2021

This identity will not be stored by the SSH agent.

$ ssh-add -l

The agent has no identities.

This is because AddKeysToAgent is set to false by default.

$ ssh -G pi-hole | grep -i AddKeysToAgent

addkeystoagent false

Update SSH client configuration.

$ cat ~/.ssh/config

Host pi-hole
  Hostname pi-hole.fishsilentcruise.space
  User milosz

Match user milosz
  IdentityFile ~/.ssh/milosz
  AddKeysToAgent yes

Ensure that AddKeysToAgent is defined.

$ ssh -G pi-hole | grep -i AddKeysToAgent

addkeystoagent true

List loaded identities.

$ ssh-add -l

The agent has no identities.

Perform SSH connection.

$ ssh pi-hole date
Enter passphrase for key '/home/milosz/.ssh/milosz': **********

Fri Oct 22 20:52:43 BST 2021

The identity will be automatically added to the SSH agent.

$ ssh-add -l

2048 SHA256:CaC957nx9z3P1vjOuRf8KaVV9enB/aypMYaaCyS5OZE /home/milosz/.ssh/milosz (RSA)

$ ssh pi-hole date

Fri Oct 22 20:53:05 BST 2021

There are more possibilities, although not useful in this specific case, so I strongly suggest reading the ssh_config manual page.

This options was added in OpenSSH 7.2 (release notes) on 2016-02-29.