Keep track of network latency using SmokePing on Debian Bullseye.
SmokePing instance configured using the following configuration
Install and configure SmokePing
Update package index.
$ sudo apt-get update
Install smokeping package, but skip recommended packages.
$ sudo apt install --no-install-recommends smokeping dnsutils curl
At first, check out the database settings as you cannot easily modify these later on. I will use default settings, which means 20 pings every 5 minutes.
$ cat /etc/smokeping/config.d/Database
*** Database ***
step = 300
pings = 20
# consfn mrhb steps total
AVERAGE 0.5 1 1008
AVERAGE 0.5 12 4320
MIN 0.5 12 4320
MAX 0.5 12 4320
AVERAGE 0.5 144 720
MAX 0.5 144 720
MIN 0.5 144 720
Update the cgiurl address.
$ cat << EOF | sudo tee /etc/smokeping/config.d/General *** General *** owner = Milosz contact = milosz@example.org mailhost = localhost # NOTE: do not put the Image Cache below cgi-bin # since all files under cgi-bin will be executed ... this is not # good for images. cgiurl = http://smoke.example.org/smokeping.cgi # specify this to get syslog logging syslogfacility = local0 # each probe is now run in its own process # disable this to revert to the old behaviour # concurrentprobes = no @include /etc/smokeping/config.d/pathnames EOF
Update imgurl path.
$ cat << EOF | sudo tee /etc/smokeping/config.d/pathnames sendmail = /usr/sbin/sendmail imgcache = /var/cache/smokeping/images imgurl = /smokeping/images datadir = /var/lib/smokeping piddir = /run/smokeping smokemail = /etc/smokeping/smokemail tmail = /etc/smokeping/tmail dyndir = /var/lib/smokeping/__cgi EOF
Define probes that will be used to check specified targets.
$ cat << EOF | sudo tee /etc/smokeping/config.d/Probes *** Probes *** + FPing binary = /usr/bin/fping +DNS binary = /usr/bin/dig +Curl binary = /usr/bin/curl +SSH binary = /usr/bin/ssh-keyscan pings = 5 EOF
Define available alerts.
$ cat << EOF | sudo tee /etc/smokeping/config.d/Alerts *** Alerts *** to = root@localhost from = smokealert@localhost +offlineatstartup type = rtt pattern = ==S,==U comment = offline at startup +hostdown_with_state type = loss pattern = >50% edgetrigger = yes comment = host down +lossdetect type = loss pattern = ==0%,==0%,==0%,==0%,==0%,>0%,>0%,>0% comment = suddenly there is packet loss +lossdetect_with_state type = loss edgetrigger = yes pattern = ==0%,==0%,==0%,==0%,==0%,>0%,>0%,>0% comment = sudden packet loss +rttdetect type = rtt pattern = <100,<100,<100,<100,<100,>100,>100,>100 comment = suddenly latency increased over 100ms +lost_5_from_20_with_state type = matcher edgetrigger = yes pattern = CheckLoss(l => 5,x => 20) comment = lost over 5 from 20 samples +rtt_avg_increased type = matcher pattern = Avgratio(historic => 20, current => 2, comparator=>'>', percentage => 150) comment = average latency from last 2 samples increased over 150% when compared to average from last 20 samples EOF
Precisely define targets using defined probes and alerts.
$ cat << EOF | sudo tee /etc/smokeping/config.d/Targets *** Targets *** menu = Top title = Network Latency Grapher probe = FPing + Example menu = Example domains title = Example domains ++ example_com menu = example.com title = Host example.com host = example.com alerts = offlineatstartup,hostdown_with_state,rttdetect,rtt_avg_increased,lossdetect ++ example_org menu = example.org title = Host example.org host = example.org alerts = offlineatstartup,hostdown_with_state,rttdetect,rtt_avg_increased,lossdetect + dns menu = DNS check title = DNS check probe = DNS ++ google menu = Google title = DNS check using Google Public DNS +++ example_org_dns_google menu = example.com title = DNS check example.com using first Google Public DNS server host = example.com pings = 5 server = 8.8.8.8 +++ example_com_dns_google menu = example.com title = DNS check example.com using second Google Public DNS server host = example.com pings = 5 server = 8.8.4.4 ++ local menu = local title = DNS check using local DNS +++ example_org_dns_local menu = example.com title = DNS check example.com using local DNS host = example.com pings = 5 server = 192.168.1.254 + ssh menu = SSH title = SSH check probe = SSH ++ example_org_ssh menu = example.org title = SSH check - example.org host = example.org keytype = rsa port = 22 alerts = hostdown_with_state,rttdetect,rtt_avg_increased + WWW menu = WWW title = WWW check probe = Curl ++ local_www menu = local title = WWW check - this website host = smoke.example.org extraargs = --user secret-user:secret-password urlformat = https://%host%/ expect = Logged in as secret-user pings = 5 alerts = offlineatstartup,hostdown_with_state ++ google_www_check menu = google.com title = WWW check - google.com host = google.com follow_redirects = yes include_redirects = yes urlformat = https://%host%/ expect = search pings = 5 alerts = offlineatstartup,hostdown_with_state,rttdetect,rtt_avg_increased EOF
Reload smokeping service.
$ sudo systemctl reload smokeping
Inspect service status.
$ systemctl status smokeping
● smokeping.service - Latency Logging and Graphing System
Loaded: loaded (/lib/systemd/system/smokeping.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-08-25 18:29:16 UTC; 5s ago
Docs: man:smokeping(1)
file:/usr/share/doc/smokeping/examples/systemd/slave_mode.conf
Process: 5273 ExecStart=/usr/sbin/smokeping --pid-dir=/run/smokeping (code=exited, status=0/SUCCESS)
Main PID: 5289 (smokeping)
Tasks: 5 (limit: 2311)
Memory: 49.8M
CPU: 283ms
CGroup: /system.slice/smokeping.service
├─5289 /usr/bin/perl /usr/sbin/smokeping --pid-dir=/run/smokeping
├─5290 /usr/sbin/smokeping [DNS]
├─5291 /usr/sbin/smokeping [SSH]
├─5292 /usr/sbin/smokeping [Curl]
└─5293 /usr/sbin/smokeping [FPing]
Inspect service logs and apply how to fix SmokePing SSH probe on Debian Buster patch if you see the following error.
$ sudo journalctl -u smokeping
-- Journal begins at Wed 2021-08-25 17:34:06 UTC, ends at Wed 2021-08-25 17:53:34 UTC. --
Aug 25 18:57:10 bullseye smokeping[2434]: Reloading configuration.
Aug 25 18:57:10 bullseye smokeping[2434]: WARNING: Hostname 'smoke.example.org' does currently not resolve to an IPv4 address
Aug 25 18:57:10 bullseye smokeping[2434]: Reloading configuration from /etc/smokeping/config failed: ERROR: output of '/usr/bin/ssh-keyscan -t dsa,rsa,rsa1 127.0.0.1' does not match (?^i:^# \S+ SSH-)
at (eval 170) line 1.
...
Install and configure uWSGI
Install uWSGI package.
$ sudo apt install --no-install-recommends uwsgi
Define smokeping application with simple basic-auth (user secret-user and password secret-password).
$ cat << EOF | sudo tee /etc/uwsgi/apps-available/smokeping.ini [uwsgi] plugins = router_basicauth, cgi route = ^/ basicauth:smokeping,secret-user:secret-password uid = www-data gid = www-data processes = 1 threads = 2 cgi = /usr/lib/cgi-bin/smokeping.cgi chown-socket = www-data:www-data EOF
Specified CGI script executes real SmokePing web-frontend.
$ cat /usr/lib/cgi-bin/smokeping.cgi
#!/bin/sh exec /usr/share/smokeping/smokeping.cgi /etc/smokeping/config
Enable application.
$ sudo ln -s /etc/uwsgi/apps-available/smokeping.ini /etc/uwsgi/apps-enabled/
Reload application server configuration.
$ sudo systemctl restart uwsgi
Install and configure nginx
Install nginx package.
$ sudo apt install --no-install-recommends nginx
Disable default Nginx site.
$ sudo unlink /etc/nginx/sites-enabled/default
Create directory for ssl certificates.
$ sudo mkdir /etc/nginx/ssl
Generate a self-signed SSL certificate for your domain.
$ sudo openssl req -subj "/commonName=smoke.example.org/" -x509 -nodes -days 730 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
Prepare nginx virtual host configuration.
$ cat << EOF | sudo tee /etc/nginx/sites-available/smoke.example.org
server {
listen 443 ssl;
server_name default;
ssl_certificate_key /etc/nginx/ssl/nginx.key;
ssl_certificate /etc/nginx/ssl/nginx.crt;
location = / {
include uwsgi_params;
uwsgi_modifier1 9;
uwsgi_pass unix:/var/run/uwsgi/app/smokeping/socket;
}
location /smokeping {
alias /usr/share/smokeping/www/;
}
}
EOF
Enable virtual host.
$ sudo ln -s /etc/nginx/sites-available/smoke.example.org /etc/nginx/sites-enabled/
Reload HTTP server configuration.
$ sudo systemctl reload nginx
It is as simple as that.
