How to monitor background process using Monit

A short time after I installed Monit, I tried to monitor a background running process that does not support daemon mode.
It is an exceptionally rare situation, but very interesting one, so after a while a came with a quite simple solution using additional shell script.

Shell script

This is a shell script used to start/stop service. It will execute /opt/service/service command in the background and manage PID file.

$ cat /usr/sbin/service.sh

#!/bin/sh
# start/stop service and create pid file for a non-daemon app
# service command
service_cmd="/opt/service/service"
# pid file location
service_pid="/var/run/service.pid"
if [ $# -eq 1 -a  "$1" = "start" ]; then
  if [ -f "$service_pid" ]; then
    kill -0 $(cat $service_pid) 2>/dev/null # check pid
    if [ $? -eq 0 ]; then
      exit 2; # process is running, exit
    else
      unlink $service_pid # process is not running
                          # remove stale pid file
    fi
  fi
  # start service in background and store process pid
  $service_cmd >/dev/null &
  echo $! > $service_pid
elif  [ $# -eq 1 -a "$1" = "stop" -a -f "$service_pid" ]; then
  kill -0 $(cat $service_pid) 2>/dev/null # check pid
  if [ $? -eq 0 ]; then
    kill $(cat $service_pid) # kill process if it is running
  fi
  unlink $service_pid
fi

This shell script is quite simple, but there are several commands that are worth your attention. I described them in two earlier blog posts: How to get PID of the background process, How to check whether the process ID still exists.

Monit configuration

Use defined PID file and code listed in the previous section to manage service using Monit.

# cat /etc/monit/conf.d/service.conf

check process service with pidfile /var/run/service.pid
  start program = "/usr/sbin/service.sh start"
  stop  program = "/usr/sbin/service.sh stop"
  depend on service_bin
  check file service_bin with path /usr/sbin/service.sh
    if failed checksum       then unmonitor
    if failed permission 755 then unmonitor
    if failed uid root       then unmonitor
    if failed gid root       then unmonitor

Sample summary information.

$ sudo monit summary
The Monit daemon 5.4 uptime: 16h 22m
Process 'service'                   Running
File 'service_bin'                  Accessible

Sample status information.

$ sudo monit status
The Monit daemon 5.4 uptime: 16h 22m
Process 'service'
  status                            Running
  monitoring status                 Monitored
  pid                               1883
  parent pid                        1
  uptime                            16h 19m
  children                          0
  memory kilobytes                  1900
  memory kilobytes total            1900
  memory percent                    0.1%
  memory percent total              0.1%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Tue, 30 Sep 2014 18:35:40
File 'service_bin'
  status                            Accessible
  monitoring status                 Monitored
  permission                        755
  uid                               0
  gid                               0
  timestamp                         Tue, 30 Sep 2014 18:35:40
  size                              1306006 B
  checksum                          bc9b654871c608bee03bf8a781c14818 (MD5)
  data collected                    Tue, 30 Sep 2014 18:35:40