Inspect current consul metrics.
Get the PID of the consul process.
$ systemctl show --property MainPID --value consul
31045
Send USR1 signal to the consul process.
sudo kill -SIGUSR1 $(systemctl show --property MainPID --value consul)
Inspect consul logs for data.
$ sudo journalctl -u consul -f
Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.last_index': 80560.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.boltdb.txstats.pageAlloc': 9900032.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.members.servers.dc-lab-1': 3.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.heap_objects': 46577.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.boltdb.numFreePages': 913.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.connect_instances.dc-lab-1.connect-native': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.ingress-gateway': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.connect_instances.dc-lab-1.ingress-gateway': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.service-resolver': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.session_ttl.active': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.applied_index': 65554.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.num_goroutines': 84.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.connect_instances.dc-lab-1.connect-proxy': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.proxy-defaults': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.boltdb.openReadTxn': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.service-intentions': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.terminating-gateway': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.service-defaults': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.kv_entries.dc-lab-1': 62.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.autopilot.failure_tolerance': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.alloc_bytes': 11396104.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.sys_bytes': 33833992.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.malloc_count': 5820461.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.nodes.dc-lab-1': 5.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.exported-services': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.service-splitter': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.boltdb.numPendingPages': 3.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.boltdb.txstats.pageCount': 2417.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.services.dc-lab-1': 3.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.connect_instances.dc-lab-1.mesh-gateway': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.service-router': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.boltdb.freelistBytes': 7344.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.service_instances.dc-lab-1': 5.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.connect_instances.dc-lab-1.terminating-gateway': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.state.config_entries.dc-lab-1.mesh': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.consul.members.clients.dc-lab-1': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.autopilot.healthy': 0.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.free_count': 5773884.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.total_gc_pause_ns': 13157574.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.runtime.total_gc_runs': 122.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][G] 'consul.raft.boltdb.freePageBytes': 3751936.000 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.memberlist.udp.sent': Count: 23 Min: 104.000 Mean: 142.391 Max: 181.000 Stddev: 38.119 Sum: 3275.000 LastUpdated: 2022-07-23 22:53:19.938807097 +0000 UTC m=+1988.209919664 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.memberlist.tcp.connect': Count: 1 Sum: 1.000 LastUpdated: 2022-07-23 22:53:11.63750088 +0000 UTC m=+1979.908613448 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.state.candidate': Count: 1 Sum: 1.000 LastUpdated: 2022-07-23 22:53:13.722373881 +0000 UTC m=+1981.993486455 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.totalReadTxn': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814405495 +0000 UTC m=+1985.085518062 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.txstats.cursorCount': Count: 2 Sum: 18.000 LastUpdated: 2022-07-23 22:53:16.814412627 +0000 UTC m=+1985.085525194 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.rpc.accept_conn': Count: 19 Sum: 19.000 LastUpdated: 2022-07-23 22:53:19.296848463 +0000 UTC m=+1987.567961030 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.memberlist.tcp.sent': Count: 1 Sum: 809.000 LastUpdated: 2022-07-23 22:53:11.637645833 +0000 UTC m=+1979.908758399 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.txstats.nodeDeref': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814415131 +0000 UTC m=+1985.085527697 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.txstats.rebalance': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814416313 +0000 UTC m=+1985.085528879 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.txstats.spill': Count: 2 Sum: 6.000 LastUpdated: 2022-07-23 22:53:16.814447296 +0000 UTC m=+1985.085559863 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.memberlist.udp.received': Count: 23 Min: 104.000 Mean: 145.739 Max: 181.000 Stddev: 37.975 Sum: 3352.000 LastUpdated: 2022-07-23 22:53:19.938643452 +0000 UTC m=+1988.209756023 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.txstats.nodeCount': Count: 2 Sum: 12.000 LastUpdated: 2022-07-23 22:53:16.814413987 +0000 UTC m=+1985.085526555 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.txstats.split': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814446205 +0000 UTC m=+1985.085558773 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][C] 'consul.raft.boltdb.txstats.write': Count: 2 Sum: 18.000 LastUpdated: 2022-07-23 22:53:16.814449934 +0000 UTC m=+1985.085562500 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.memberlist.pushPullNode': Count: 1 Sum: 1.340 LastUpdated: 2022-07-23 22:53:11.638359857 +0000 UTC m=+1979.909472424 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.raft.boltdb.txstats.rebalanceTime': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814443064 +0000 UTC m=+1985.085555636 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.serf.queue.Intent': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:11.95907866 +0000 UTC m=+1980.230191228 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.serf.queue.Event': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:11.959109924 +0000 UTC m=+1980.230222492 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.runtime.gc_pause_ns': Count: 1 Sum: 85561.000 LastUpdated: 2022-07-23 22:53:15.250622957 +0000 UTC m=+1983.521735524 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.memberlist.gossip': Count: 70 Min: 0.006 Mean: 0.019 Max: 0.063 Stddev: 0.007 Sum: 1.304 LastUpdated: 2022-07-23 22:53:19.917414952 +0000 UTC m=+1988.188527519 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.serf.coordinate.adjustment-ms': Count: 12 Min: 0.130 Mean: 0.333 Max: 0.402 Stddev: 0.095 Sum: 3.995 LastUpdated: 2022-07-23 22:53:19.918126991 +0000 UTC m=+1988.189239557 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.memberlist.probeNode': Count: 12 Min: 0.488 Mean: 0.684 Max: 0.795 Stddev: 0.081 Sum: 8.213 LastUpdated: 2022-07-23 22:53:19.918129767 +0000 UTC m=+1988.189242334 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.raft.boltdb.txstats.spillTime': Count: 2 Min: 0.016 Mean: 0.018 Max: 0.019 Stddev: 0.002 Sum: 0.036 LastUpdated: 2022-07-23 22:53:16.814448764 +0000 UTC m=+1985.085561332 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.raft.boltdb.txstats.writeTime': Count: 2 Min: 6.659 Mean: 7.145 Max: 7.631 Stddev: 0.687 Sum: 14.290 LastUpdated: 2022-07-23 22:53:16.814451344 +0000 UTC m=+1985.085563910 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.serf.queue.Query': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:11.959120713 +0000 UTC m=+1980.230233281 Jul 23 22:53:27 consul-2 consul[31045]: [2022-07-23 22:53:10 +0000 UTC][S] 'consul.raft.candidate.electSelf': Count: 1 Sum: 1.474 LastUpdated: 2022-07-23 22:53:13.726320218 +0000 UTC m=+1981.997432793
Use sed
to extract metrics data.
$ sudo journalctl -u consul -f | sed -n -e "s/.*\[.*\]\[[CS]\] \(.*\)/\1/p"
'consul.memberlist.udp.sent': Count: 23 Min: 104.000 Mean: 142.391 Max: 181.000 Stddev: 38.119 Sum: 3275.000 LastUpdated: 2022-07-23 22:53:19.938807097 +0000 UTC m=+1988.209919664 'consul.memberlist.tcp.connect': Count: 1 Sum: 1.000 LastUpdated: 2022-07-23 22:53:11.63750088 +0000 UTC m=+1979.908613448 'consul.raft.state.candidate': Count: 1 Sum: 1.000 LastUpdated: 2022-07-23 22:53:13.722373881 +0000 UTC m=+1981.993486455 'consul.raft.boltdb.totalReadTxn': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814405495 +0000 UTC m=+1985.085518062 'consul.raft.boltdb.txstats.cursorCount': Count: 2 Sum: 18.000 LastUpdated: 2022-07-23 22:53:16.814412627 +0000 UTC m=+1985.085525194 'consul.rpc.accept_conn': Count: 19 Sum: 19.000 LastUpdated: 2022-07-23 22:53:19.296848463 +0000 UTC m=+1987.567961030 'consul.memberlist.tcp.sent': Count: 1 Sum: 809.000 LastUpdated: 2022-07-23 22:53:11.637645833 +0000 UTC m=+1979.908758399 'consul.raft.boltdb.txstats.nodeDeref': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814415131 +0000 UTC m=+1985.085527697 'consul.raft.boltdb.txstats.rebalance': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814416313 +0000 UTC m=+1985.085528879 'consul.raft.boltdb.txstats.spill': Count: 2 Sum: 6.000 LastUpdated: 2022-07-23 22:53:16.814447296 +0000 UTC m=+1985.085559863 'consul.memberlist.udp.received': Count: 23 Min: 104.000 Mean: 145.739 Max: 181.000 Stddev: 37.975 Sum: 3352.000 LastUpdated: 2022-07-23 22:53:19.938643452 +0000 UTC m=+1988.209756023 'consul.raft.boltdb.txstats.nodeCount': Count: 2 Sum: 12.000 LastUpdated: 2022-07-23 22:53:16.814413987 +0000 UTC m=+1985.085526555 'consul.raft.boltdb.txstats.split': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814446205 +0000 UTC m=+1985.085558773 'consul.raft.boltdb.txstats.write': Count: 2 Sum: 18.000 LastUpdated: 2022-07-23 22:53:16.814449934 +0000 UTC m=+1985.085562500 'consul.memberlist.pushPullNode': Count: 1 Sum: 1.340 LastUpdated: 2022-07-23 22:53:11.638359857 +0000 UTC m=+1979.909472424 'consul.raft.boltdb.txstats.rebalanceTime': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:16.814443064 +0000 UTC m=+1985.085555636 'consul.serf.queue.Intent': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:11.95907866 +0000 UTC m=+1980.230191228 'consul.serf.queue.Event': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:11.959109924 +0000 UTC m=+1980.230222492 'consul.runtime.gc_pause_ns': Count: 1 Sum: 85561.000 LastUpdated: 2022-07-23 22:53:15.250622957 +0000 UTC m=+1983.521735524 'consul.memberlist.gossip': Count: 70 Min: 0.006 Mean: 0.019 Max: 0.063 Stddev: 0.007 Sum: 1.304 LastUpdated: 2022-07-23 22:53:19.917414952 +0000 UTC m=+1988.188527519 'consul.serf.coordinate.adjustment-ms': Count: 12 Min: 0.130 Mean: 0.333 Max: 0.402 Stddev: 0.095 Sum: 3.995 LastUpdated: 2022-07-23 22:53:19.918126991 +0000 UTC m=+1988.189239557 'consul.memberlist.probeNode': Count: 12 Min: 0.488 Mean: 0.684 Max: 0.795 Stddev: 0.081 Sum: 8.213 LastUpdated: 2022-07-23 22:53:19.918129767 +0000 UTC m=+1988.189242334 'consul.raft.boltdb.txstats.spillTime': Count: 2 Min: 0.016 Mean: 0.018 Max: 0.019 Stddev: 0.002 Sum: 0.036 LastUpdated: 2022-07-23 22:53:16.814448764 +0000 UTC m=+1985.085561332 'consul.raft.boltdb.txstats.writeTime': Count: 2 Min: 6.659 Mean: 7.145 Max: 7.631 Stddev: 0.687 Sum: 14.290 LastUpdated: 2022-07-23 22:53:16.814451344 +0000 UTC m=+1985.085563910 'consul.serf.queue.Query': Count: 2 Sum: 0.000 LastUpdated: 2022-07-23 22:53:11.959120713 +0000 UTC m=+1980.230233281 'consul.raft.candidate.electSelf': Count: 1 Sum: 1.474 LastUpdated: 2022-07-23 22:53:13.726320218 +0000 UTC m=+1981.997432793
You can also use API to access metrics data.
$ curl "http://127.0.0.1:8500/v1/agent/metrics?pretty"
{ "Timestamp": "2022-07-23 22:57:40 +0000 UTC", "Gauges": [ { "Name": "consul.autopilot.failure_tolerance", "Value": 0, "Labels": {} }, { "Name": "consul.autopilot.healthy", "Value": 0, "Labels": {} }, { "Name": "consul.consul.members.clients", "Value": 0, "Labels": { "datacenter": "dc-lab-1" } }, { "Name": "consul.consul.members.servers", "Value": 3, "Labels": { "datacenter": "dc-lab-1" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "exported-services" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "ingress-gateway" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "mesh" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "proxy-defaults" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "service-defaults" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "service-intentions" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "service-resolver" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "service-router" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "service-splitter" } }, { "Name": "consul.consul.state.config_entries", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "terminating-gateway" } }, { "Name": "consul.consul.state.connect_instances", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "connect-native" } }, { "Name": "consul.consul.state.connect_instances", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "connect-proxy" } }, { "Name": "consul.consul.state.connect_instances", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "ingress-gateway" } }, { "Name": "consul.consul.state.connect_instances", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "mesh-gateway" } }, { "Name": "consul.consul.state.connect_instances", "Value": 0, "Labels": { "datacenter": "dc-lab-1", "kind": "terminating-gateway" } }, { "Name": "consul.consul.state.kv_entries", "Value": 62, "Labels": { "datacenter": "dc-lab-1" } }, { "Name": "consul.consul.state.nodes", "Value": 5, "Labels": { "datacenter": "dc-lab-1" } }, { "Name": "consul.consul.state.service_instances", "Value": 5, "Labels": { "datacenter": "dc-lab-1" } }, { "Name": "consul.consul.state.services", "Value": 3, "Labels": { "datacenter": "dc-lab-1" } }, { "Name": "consul.raft.applied_index", "Value": 65554, "Labels": {} }, { "Name": "consul.raft.boltdb.freePageBytes", "Value": 3751936, "Labels": {} }, { "Name": "consul.raft.boltdb.freelistBytes", "Value": 7344, "Labels": {} }, { "Name": "consul.raft.boltdb.numFreePages", "Value": 913, "Labels": {} }, { "Name": "consul.raft.boltdb.numPendingPages", "Value": 3, "Labels": {} }, { "Name": "consul.raft.boltdb.openReadTxn", "Value": 0, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.pageAlloc", "Value": 11300864, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.pageCount", "Value": 2759, "Labels": {} }, { "Name": "consul.raft.last_index", "Value": 80560, "Labels": {} }, { "Name": "consul.runtime.alloc_bytes", "Value": 20196600, "Labels": {} }, { "Name": "consul.runtime.free_count", "Value": 6344014, "Labels": {} }, { "Name": "consul.runtime.heap_objects", "Value": 77448, "Labels": {} }, { "Name": "consul.runtime.malloc_count", "Value": 6421462, "Labels": {} }, { "Name": "consul.runtime.num_goroutines", "Value": 84, "Labels": {} }, { "Name": "consul.runtime.sys_bytes", "Value": 34096136, "Labels": {} }, { "Name": "consul.runtime.total_gc_pause_ns", "Value": 14784511, "Labels": {} }, { "Name": "consul.runtime.total_gc_runs", "Value": 137, "Labels": {} }, { "Name": "consul.session_ttl.active", "Value": 0, "Labels": {} } ], "Points": [], "Counters": [ { "Name": "consul.client.rpc", "Count": 2, "Rate": 0.2, "Sum": 2, "Min": 1, "Max": 1, "Mean": 1, "Stddev": 0, "Labels": {} }, { "Name": "consul.memberlist.tcp.accept", "Count": 1, "Rate": 0.1, "Sum": 1, "Min": 1, "Max": 1, "Mean": 1, "Stddev": 0, "Labels": {} }, { "Name": "consul.memberlist.tcp.sent", "Count": 1, "Rate": 80.5, "Sum": 805, "Min": 805, "Max": 805, "Mean": 805, "Stddev": 0, "Labels": {} }, { "Name": "consul.memberlist.udp.received", "Count": 24, "Rate": 345.6, "Sum": 3456, "Min": 104, "Max": 181, "Mean": 144, "Stddev": 38.1051177665153, "Labels": {} }, { "Name": "consul.memberlist.udp.sent", "Count": 24, "Rate": 345.6, "Sum": 3456, "Min": 104, "Max": 181, "Mean": 144, "Stddev": 38.1051177665153, "Labels": {} }, { "Name": "consul.raft.boltdb.totalReadTxn", "Count": 2, "Rate": 0.1, "Sum": 1, "Min": 0, "Max": 1, "Mean": 0.5, "Stddev": 0.7071067811865476, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.cursorCount", "Count": 2, "Rate": 2, "Sum": 20, "Min": 9, "Max": 11, "Mean": 10, "Stddev": 1.4142135623730951, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.nodeCount", "Count": 2, "Rate": 1.2, "Sum": 12, "Min": 6, "Max": 6, "Mean": 6, "Stddev": 0, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.nodeDeref", "Count": 2, "Rate": 0, "Sum": 0, "Min": 0, "Max": 0, "Mean": 0, "Stddev": 0, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.rebalance", "Count": 2, "Rate": 0, "Sum": 0, "Min": 0, "Max": 0, "Mean": 0, "Stddev": 0, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.spill", "Count": 2, "Rate": 0.6, "Sum": 6, "Min": 3, "Max": 3, "Mean": 3, "Stddev": 0, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.split", "Count": 2, "Rate": 0, "Sum": 0, "Min": 0, "Max": 0, "Mean": 0, "Stddev": 0, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.write", "Count": 2, "Rate": 1.8, "Sum": 18, "Min": 9, "Max": 9, "Mean": 9, "Stddev": 0, "Labels": {} }, { "Name": "consul.raft.state.candidate", "Count": 1, "Rate": 0.1, "Sum": 1, "Min": 1, "Max": 1, "Mean": 1, "Stddev": 0, "Labels": {} }, { "Name": "consul.rpc.accept_conn", "Count": 19, "Rate": 1.9, "Sum": 19, "Min": 1, "Max": 1, "Mean": 1, "Stddev": 0, "Labels": {} } ], "Samples": [ { "Name": "consul.acl.ResolveToken", "Count": 1, "Rate": 0.004297799989581108, "Sum": 0.04297799989581108, "Min": 0.04297799989581108, "Max": 0.04297799989581108, "Mean": 0.04297799989581108, "Stddev": 0, "Labels": {} }, { "Name": "consul.api.http", "Count": 1, "Rate": 0.05300869941711426, "Sum": 0.5300869941711426, "Min": 0.5300869941711426, "Max": 0.5300869941711426, "Mean": 0.5300869941711426, "Stddev": 0, "Labels": { "method": "GET", "path": "v1_agent_metrics" } }, { "Name": "consul.api.http", "Count": 1, "Rate": 1246.4619140625, "Sum": 12464.619140625, "Min": 12464.619140625, "Max": 12464.619140625, "Mean": 12464.619140625, "Stddev": 0, "Labels": { "method": "GET", "path": "v1_agent_monitor" } }, { "Name": "consul.memberlist.gossip", "Count": 70, "Rate": 0.1340316001791507, "Sum": 1.340316001791507, "Min": 0.0066559999249875546, "Max": 0.03837500140070915, "Mean": 0.019147371454164386, "Stddev": 0.005150694713146948, "Labels": {} }, { "Name": "consul.memberlist.probeNode", "Count": 12, "Rate": 0.8715510934591293, "Sum": 8.715510934591293, "Min": 0.4353649914264679, "Max": 0.8789029717445374, "Mean": 0.7262925778826078, "Stddev": 0.1086966914278882, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.rebalanceTime", "Count": 2, "Rate": 0, "Sum": 0, "Min": 0, "Max": 0, "Mean": 0, "Stddev": 0, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.spillTime", "Count": 2, "Rate": 0.005971099808812141, "Sum": 0.059710998088121414, "Min": 0.01712699979543686, "Max": 0.042583998292684555, "Mean": 0.029855499044060707, "Stddev": 0.018000816266059597, "Labels": {} }, { "Name": "consul.raft.boltdb.txstats.writeTime", "Count": 2, "Rate": 1.4999948024749756, "Sum": 14.999948024749756, "Min": 7.315351963043213, "Max": 7.684596061706543, "Mean": 7.499974012374878, "Stddev": 0.2610950060779553, "Labels": {} }, { "Name": "consul.raft.candidate.electSelf", "Count": 1, "Rate": 0.14611899852752686, "Sum": 1.4611899852752686, "Min": 1.4611899852752686, "Max": 1.4611899852752686, "Mean": 1.4611899852752686, "Stddev": 0, "Labels": {} }, { "Name": "consul.serf.coordinate.adjustment-ms", "Count": 12, "Rate": 0.2115220993757248, "Sum": 2.115220993757248, "Min": 0.13530999422073364, "Max": 0.24645300209522247, "Mean": 0.17626841614643732, "Stddev": 0.035050204255583875, "Labels": {} }, { "Name": "consul.serf.queue.Event", "Count": 2, "Rate": 0, "Sum": 0, "Min": 0, "Max": 0, "Mean": 0, "Stddev": 0, "Labels": {} }, { "Name": "consul.serf.queue.Intent", "Count": 2, "Rate": 0, "Sum": 0, "Min": 0, "Max": 0, "Mean": 0, "Stddev": 0, "Labels": {} }, { "Name": "consul.serf.queue.Query", "Count": 2, "Rate": 0, "Sum": 0, "Min": 0, "Max": 0, "Mean": 0, "Stddev": 0, "Labels": {} } ] }
This format makes it easy to parse.
$ curl --silent "http://127.0.0.1:8500/v1/agent/metrics?pretty" | \ jq '.Gauges[] | select(.Name=="consul.consul.members.servers") | .Value'
3