Prometheus Client Service Output Plugin [[outputs.prometheus_client]] no output for the defined scripts with [[inputs.exec]]

Telegraf
,
1

Hi,

Issue:
I'm searching a way to have the metrics collected by telegraf with the help of scripts and after this to expose on localhost:9373/metrics to have it in Prometheus. Kindly does someone have any idea why I don't have the metrics even if I've modified the scripts to give: 0, 1 instead of UP/ DOWN.

System:
telegraf-1.6.0-1.x86_64
influxdb-1.5.2-1.x86_64

Telegraf.conf:

[[outputs.prometheus_client]]
listen = “:9373”
path = “/metrics”
expiration_interval = “25s”
string_as_label = false

[[inputs.exec]]
commands = [“/etc/telegraf/scripts/os_version.sh”]
name_override = “os_version”
data_format = “value”
data_type = “string”

[[inputs.exec]]
commands = [“/etc/telegraf/scripts/os_kernel.sh”]
name_override = “os_kernel”
data_format = “value”
data_type = “string”

[[inputs.exec]]
commands = [“/etc/telegraf/scripts/connector_version.sh”]
name_override = “connector_version”
data_format = “value”
data_type = “string”

[[inputs.exec]]
commands = [“/etc/telegraf/scripts/connector_status.sh”]
name_override = “connector_status”
data_format = “value”
data_type = “string”

[[inputs.exec]]
commands = [“/etc/telegraf/scripts/connector_components.sh”]
name_override = “connector_components_status”
data_format = “value”
data_type = “string”

[[inputs.exec]]
commands = [“/etc/telegraf/scripts/connector_components_version.sh”]
name_override = “connector_components_version”
data_format = “value”
data_type = “string”

Telegraf --test output:
[root@server1]# /usr/bin/telegraf --config /etc/telegraf/telegraf.conf --test

  • Plugin: inputs.exec, Collection 1

os_version,host=server1 value=“CentOS Linux 7.4.1708” 1524409762000000000

  • Plugin: inputs.exec, Collection 1

os_kernel,host=server1 value=“3.10.0-514.6.2.el7.x86_64” 1524409762000000000

  • Plugin: inputs.exec, Collection 1

connector_version,host=server1 value=“89” 1524409762000000000

  • Plugin: inputs.exec, Collection 1

connector_status,host=server1 value=“1” 1524409762000000000

  • Plugin: inputs.exec, Collection 1

connector_components_status,host=server1 value=“jedisConnectionOne_status=1,redisCacheConnectionOne_status=1,cazMqtt_status=1,zoMqtt_status=1,rabbitMQ_status=1” 1524409762000000000

  • Plugin: inputs.exec, Collection 1

connector_components_version,host=server1 value=“jedisConnectionOne_version=3.2.10,redisCacheConnectionOne_version=3.2.10,rabbitMQ_version=3.6.12” 1524409762000000000

[root@server1]# /usr/bin/curl -s localhost:9373/metrics | grep -v “^#”
go_gc_duration_seconds{quantile=“0”} 0.000198659
go_gc_duration_seconds{quantile=“0.25”} 0.000198659
go_gc_duration_seconds{quantile=“0.5”} 0.000199599
go_gc_duration_seconds{quantile=“0.75”} 0.000199599
go_gc_duration_seconds{quantile=“1”} 0.000199599
go_gc_duration_seconds_sum 0.000398258
go_gc_duration_seconds_count 2
go_goroutines 22
go_memstats_alloc_bytes 4.560744e+06
go_memstats_alloc_bytes_total 6.054048e+06
go_memstats_buck_hash_sys_bytes 1.447994e+06
go_memstats_frees_total 14710
go_memstats_gc_sys_bytes 442368
go_memstats_heap_alloc_bytes 4.560744e+06
go_memstats_heap_idle_bytes 802816
go_memstats_heap_inuse_bytes 5.849088e+06
go_memstats_heap_objects 13049
go_memstats_heap_released_bytes 0
go_memstats_heap_sys_bytes 6.651904e+06
go_memstats_last_gc_time_seconds 1.5244097570822785e+09
go_memstats_lookups_total 138
go_memstats_mallocs_total 27759
go_memstats_mcache_inuse_bytes 3472
go_memstats_mcache_sys_bytes 16384
go_memstats_mspan_inuse_bytes 47120
go_memstats_mspan_sys_bytes 65536
go_memstats_next_gc_bytes 8.387344e+06
go_memstats_other_sys_bytes 753854
go_memstats_stack_inuse_bytes 688128
go_memstats_stack_sys_bytes 688128
go_memstats_sys_bytes 1.0066168e+07
process_cpu_seconds_total 0.09
process_max_fds 1024
process_open_fds 9
process_resident_memory_bytes 1.6125952e+07
process_start_time_seconds 1.52440975681e+09
process_virtual_memory_bytes 2.6406912e+08

Any recommendation/ idea?

Kind Regards,

2

Prometheus doesn’t support string values, but it should work if switch the numeric exec plugins to use data_type = "float". For the strings like “3.10.0-514.6.2.el7.x86_64” it would probably make sense to turn these into tags, but to do this you would need to have your scripts output in “influx” line protocol format and you will need still need at least one numerical value.

1 Like
3

Many thanks Daniel.
I’ve didn’t paid attention to data_type, now I’ve figured out this aspect and yes Prometheus doesn’t know to work proper with strings.

[root@server1 ~]# /usr/bin/telegraf --config /etc/telegraf/telegraf.conf --test

  • Plugin: inputs.exec, Collection 1

connector_version,host=server1 value=90 1524507664000000000

  • Plugin: inputs.exec, Collection 1

connector_status,host=server1 value=1 1524507664000000000

[root@server1 ~]# /usr/bin/curl -s localhost:9373/romel | grep -v “^#”
connector_status{host=“server1”} 1
connector_version{host=“server1”} 90