Increase the system-wide number of available file handles (open files).

Recently, I could not connect to an example.org server using OpenSSH client as it just stopped responding instead of giving me a shell prompt.

$ ssh example.org -vv

[...]
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-s2a2-nistp256 SHA256:02f7XfMGsmlj94TAm9LqxS1rOHcL0GUc1Phwz0gXkigw
debug1: Host 'example.org' is known and matches the ECDSA host key.
debug1: Found key in /home/milosz/.ssh/known_hosts:885
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /home/milosz/.ssh/id_rsa (0x55ed2a3ecee0)
debug2: key: /home/milosz/.ssh/id_dsa ((nil))
debug2: key: /home/milosz/.ssh/id_ecdsa ((nil))
debug2: key: /home/milosz/.ssh/id_ed25519 ((nil))
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000)
debug1: Unspecified GSS failure.  Minor code may provide more information
No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000)
debug2: we did not send a packet, disable method
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:wqGEmK0hE2F0rQaTEg1v0sVXGIN913A8aGZEd3Zmz6QQ /home/milosz/.ssh/id_rsa
debug2: we sent a publickey packet, wait for reply
debug1: Server accepts key: pkalg rsa-sha2-512 blen 535
debug2: input_userauth_pk_ok: fp SHA256:wqGEmK0hE2F0rQaTEg1v0sVXGIN913A8aGZEd3Zmz6QQ
Enter passphrase for key '/home/milosz/.ssh/id_rsa': ****************
debug1: Authentication succeeded (publickey).
Authenticated to example.org ([172.16.0.18]:22).
debug1: channel 0: new [client-session]
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
[There is nothing more.]
</rsa-sha2-256,rsa-sha2-512>

The real culprit was the exceeded number of available file handles.

$ dmesg

[...]
[5512385.576209] VFS: file-max limit 100000 reached
[5512385.635161] VFS: file-max limit 100000 reached
[5512385.650384] VFS: file-max limit 100000 reached
[...]

Other running services were affected as well.

[2020-01-17T23:56:48,463][WARN ][i.n.c.DefaultChannelPipeline] [NJj8BTd] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.io.IOException: Too many open files in system
	at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[?:?]
	at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source) ~[?:?]
	at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source) ~[?:?]
	at io.netty.util.internal.SocketUtils$5.run(SocketUtils.java:110) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.util.internal.SocketUtils$5.run(SocketUtils.java:107) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
	at io.netty.util.internal.SocketUtils.accept(SocketUtils.java:107) ~[netty-common-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.socket.nio.NioServerSocketChannel.doReadMessages(NioServerSocketChannel.java:143) ~[netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:75) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
	at java.lang.Thread.run(Unknown Source) [?:?]

Display the system-wide number of available file handles.

$ sudo sysctl fs.file-max
fs.file-max = 100000

Display the number of used file-handles. The first number denotes allocated file-handles, the second allocated, but unused file handles, and the third one the maximum number of file handles.

$ sudo sysctl fs.file-nr
fs.file-nr = 100128     0       100000

Increase the maximum number of file handles.

$ sudo sysctl -w fs.file-max=500000
fs.file-max = 500000

Make this change persistent across a reboot.

$ echo "fs.file-max = 500000" | sudo tee /etc/sysctl.d/local-file-max.conf

Verify the system-wide number of available file handles.

$ sudo sysctl fs.file-max
fs.file-max = 500000

Display the top eight processes using the most file handles to determine the next step.

$ lsof | awk '{print $1}' | sort | uniq -c | sort -r | head -8
  89712 mysqld
  61587 java
  14475 varnishd
   1102 proxysql
    760 php-fpm
    688 redis-server
    410 nginx
    348 tuned

Additional information

Please read documentation for /proc/sys/fs/* for more information.