[solved] Telegraf error converting ASCII presentation of floats

Telegraf
1

Hello,
I have a mix of MQTT payloads that are numbers represented by the binary values and numbers represented by the ASCII. To make it more difficult, I have both floats and integers encoded in ASCII, coming in as MQTT payloads.

For example:
nvme1@nvme1:~$ mosquitto_sub -h 192.168.2.24 -v -t ‘#’
/yyy/nano_01/c3be0687522e/mac c3be0687522e
/yyy/nano_01/c3be0687522e/rssi -84 <-------this is an integer
/yyy/nano_01/c3be0687522e/volt 2.95 <------- this is actually ASCII !!!
/yyy/nano_01/c3be0687522e/tmr 612
/yyy/nano_01/c3be0687522e/xmit_cnt 7
/yyy/nano_01/c3be0687522e/TMP/p 68.11 <---- this is ASCII as well !!!

When I have Telegraf’s MQTT plugin try to put these topics into InfluxDB, I get these errors:

2017-11-10T06:21:45Z E! Error in plugin [inputs.mqtt_consumer]: E! MQTT Parse Error
message: 68.22
error: strconv.Atoi: parsing “68.22”: invalid syntax
2017-11-10T06:21:50Z D! Output [influxdb] buffer fullness: 5 / 10000 metrics.
2017-11-10T06:21:50Z D! Output [influxdb] wrote batch of 5 metrics in 6.208344ms
2017-11-10T06:21:55Z E! Error in plugin [inputs.mqtt_consumer]: E! MQTT Parse Error
message: 2.97
error: strconv.Atoi: parsing “2.97”: invalid syntax

That’s understandable. It’s smart enough to know it’s ASCII and is trying to use Atoi to do the string convert to values. But it fails on the non-integer ASCII data. It works on integer ASCII values.

I know which topics are ASCII floats and which as ASCII Integers. I can specify it, if you provide a configuration option to individually specify which topics should be processed with what values to expect.

Or, can it be smarter about using atoi or atof?

Here’s my input configuration. Do you want to see my output config for influxDB? Does that have any bearing on this issue? I don’t have a good understanding of input formats - I selected value because data_format = “influx” gave me parse errors on everything.

# Read metrics from MQTT topic(s)
[[inputs.mqtt_consumer]]
  ## MQTT broker URLs to be used. The format should be scheme://host:port,
  ## schema can be tcp, ssl, or ws.
  servers = ["tcp://localhost:1883"]
  ## MQTT QoS, must be 0, 1, or 2
  qos = 0
  ## Connection timeout for initial connection in seconds
  #connection_timeout = "30s"

  ## Topics to subscribe to
  topics = [
    "telegraf/host01/cpu",
    "telegraf/+/mem",
    "sensors/#",
    "/yyy/nano_01/c3be0687522e/rssi",
    "/yyy/nano_01/c3be0687522e/rssi/#",
    "/yyy/nano_01/c3be0687522e/volt/#",
    "/yyy/nano_01/c3be0687522e/TMP/#",
    "/yyy/+/+/mag/#",
  ]


  persistent_session = false
  client_id = ""

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  #data_format = "influx"
  data_format = "value"
2

You can use the data_type parameter to set the type to parse, and then setup one mqtt_consumer for each type:

[[inputs.mqtt_consumer]]
  # .. snip ..
  topics = [
    "/string/topics/#"
  ]
  data_format = "value"
  data_type = "string"

[[inputs.mqtt_consumer]]
  # .. snip ..
  topics = [
    "/int/topics/#"
  ]
  data_format = "value"
  data_type = "int"
3

Thanks, that did fix it. Closing the loop for anyone else, it seems you have to use the “name_override” in additional inputs.mqtt_consumer to give it a different name. So in addition to the telegraf conf above for integers, I added this for floats:

# Read metrics from MQTT topic(s) FLOATS
[[inputs.mqtt_consumer]]
  name_override = "mqtt_consumer_floats"
  servers = ["tcp://localhost:1883"]
  qos = 0
  #connection_timeout = "30s"
  topics = [
    "/+/+/+/volt/#",
    "/+/+/+/TMP/#",
    "/+/+/+/HUM/#",
  ]
  persistent_session = false
  client_id = ""
  # insecure_skip_verify = false
  #data_format = "influx"
  data_format = "value"
  data_type = "float"

It does break my MQTT topic naming scheme a bit. I can’t rely on a # to just catch everything coming over MQTT. The MQTT plugin for Telegraf seems to need to know data types before hand. Is this something that can be added as a feature?

4

You might also want to consider using data_format = "influx", this would give you a lot more control over how the metrics are created.