Keeping SSH authentication agent in sync across multiple servers and terminal multiplexers is not an easy task, but can be solved by using consistent socket names used to connect to the SSH authentication agent.

Create ~/.bash_ssh-agent shell script that will manage local and remote SSH authentication agent.

#!/bin/bash
# start ssh-agent using specified socket or update shell variables
# ssh-agent parameters
SSH_AUTH_SOCKET="${HOME}/.ssh/agent_socket"
SSH_LIFE=3600
if [ -z "$SSH_TTY" ]; then
  # local terminal
  SSH_AGENT_PID="$(cat ${HOME}/.ssh/agent_pid)"
  ps --pid $SSH_AGENT_PID -o comm= 1>/dev/null 2>/dev/null
  if [ "$?" -ne "0" ]; then
    # start ssh_agent
    if [ -e "$SSH_AUTH_SOCKET" ]; then
      unlink $SSH_AUTH_SOCKET
    fi
    eval $(ssh-agent -s -a $SSH_AUTH_SOCKET -t $SSH_LIFE)
    if [ ! -d "${HOME}/.ssh" ]; then
      mkdir ${HOME}/.ssh
    fi
    echo $SSH_AGENT_PID > ${HOME}/.ssh/agent_pid
  else
    # agent is already running, update variables
    export SSH_AGENT_PID
    export SSH_AUTH_SOCK="$SSH_AUTH_SOCKET"
  fi
else
  # remote terminal
  if [ -n "$SSH_AUTH_SOCK" ] && [ "$SSH_AUTH_SOCK" != "$SSH_AUTH_SOCKET" ]; then
    ln -sf $SSH_AUTH_SOCK $SSH_AUTH_SOCKET
    export SSH_AUTH_SOCK="$SSH_AUTH_SOCKET"
  fi
fi

Update ~/.bashrc on local and remote to source created shell script.

[...]
# start ssh-agent using specified socket or update shell variables
[ -f ~/.bash_ssh-agent ] && . ~/.bash_ssh-agent
[...]

This is awesome!