MQTT Telegraf connected but no Data

,
1

Hello all,

I have configured the MQTT consumer Telegraf as follows.

   # Configuration for telegraf agent
[agent]
  ## Default data collection interval for all inputs
  interval = "10s"
  ## Rounds collection interval to 'interval'
  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
  round_interval = true

  ## Telegraf will send metrics to outputs in batches of at most
  ## metric_batch_size metrics.
  ## This controls the size of writes that Telegraf sends to output plugins.
  metric_batch_size = 1000

  ## For failed writes, telegraf will cache metric_buffer_limit metrics for each
  ## output, and will flush this buffer on a successful write. Oldest metrics
  ## are dropped first when this buffer fills.
  ## This buffer only fills when writes fail to output plugin(s).
  metric_buffer_limit = 10000

  ## Collection jitter is used to jitter the collection by a random amount.
  ## Each plugin will sleep for a random time within jitter before collecting.
  ## This can be used to avoid many plugins querying things like sysfs at the
  ## same time, which can have a measurable effect on the system.
  collection_jitter = "0s"

  ## Default flushing interval for all outputs. Maximum flush_interval will be
  ## flush_interval + flush_jitter
  flush_interval = "10s"
  ## Jitter the flush interval by a random amount. This is primarily to avoid
  ## large write spikes for users running a large number of telegraf instances.
  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
  flush_jitter = "0s"

  ## By default or when set to "0s", precision will be set to the same
  ## timestamp order as the collection interval, with the maximum being 1s.
  ##   ie, when interval = "10s", precision will be "1s"
  ##       when interval = "250ms", precision will be "1ms"
  ## Precision will NOT be used for service inputs. It is up to each individual
  ## service input to set the timestamp at the appropriate precision.
  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
  precision = ""

  ## Logging configuration:
  ## Run telegraf with debug log messages.
  debug = false
  ## Run telegraf in quiet mode (error log messages only).
  quiet = false
  ## Specify the log file name. The empty string means to log to stderr.
  logfile = ""

  ## Override default hostname, if empty use os.Hostname()
  hostname = ""
  ## If set to true, do no set the "host" tag in the telegraf agent.
  omit_hostname = false

[[outputs.influxdb_v2]]
  ## The URLs of the InfluxDB cluster nodes.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  ## urls exp: http://127.0.0.1:8086
  urls = ["http://10.100.124.6:8086"]

  ## Token for authentication.
  token = "$INFLUX_TOKEN"

  ## Organization is the name of the organization you wish to write to; must exist.
  organization = "SV-Alzey"

  ## Destination bucket to write into.
  bucket = "bucket"

[[inputs.cpu]]
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics.
  collect_cpu_time = false
  ## If true, compute and report the sum of all non-idle CPU states.
  report_active = false

[[inputs.disk]]
  ## By default stats will be gathered for all mount points.
  ## Set mount_points will restrict the stats to only the specified mount points.
  # mount_points = ["/"]
  ## Ignore mount points by filesystem type.
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]
[[inputs.mem]]
[[inputs.net]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]

[[inputs.mqtt_consumer]]
#
# MQTT broker URLs to be used. The format is scheme://host:port, schema can be tcp, ssl, or ws.
  servers = ["tcp://10.100.124.24:1883"]
#
# Topics to subscribe to.
  topics = [
      "#"
      ]
#
 qos = 1
 persistent_session = true
 client_id = "Telegraf_Client"
# Username and password.
  username = "MQTT"
  password = "password"
#
# Needed only if your payload type is string, since Telegraf does not forward data of this type by default.
#    json_string_fields = ["uplink_message_frm_payload"]
#
# Define the message format.
  data_format = "json"

On the broker I can see that the telegraph connects, but I cannot see any data.
With another MQTT client, I tried that the broker also sends data, it sends data.
But why don’t I see it in Explore?

Thanks for your help

Matthias

2

I have the output set to file and get this:

2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:5: "3000"
2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "2,798"
2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "0,494"
2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "2,009"
2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:5: "2393"
2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "235,6"
2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "236,2"
2020-12-15T10:14:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "239,4"
2020-12-15T10:14:32Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:5: "1384"
2020-12-15T10:14:45Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:2: "0"
3

Hello @matze1708,
Can you please also write your data to stdout?

[[outputs.file]]
  ## Files to write to, "stdout" is a specially handled file.
  files = ["stdout"]

  ## Data format to output.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
  data_format = "influx"

And share the output? We want to make sure that it matches the line protocol.

Or use the telegraf test command?
./telegraf -config telegraf.conf -test

Thank you.

4

Good Morning,

Thanks for the help,
here is the extract from the file with debug output.
I think I’ve been able to track it down a bit more.

I get from the MQTT broker float values with comma 230.5 but the database can not handle it, it would need 230.5 can this be adjusted somehow?

2020-12-16T07:11:04Z I! Starting Telegraf 1.16.3
2020-12-16T07:11:04Z I! Loaded inputs: mqtt_consumer
2020-12-16T07:11:04Z I! Loaded aggregators:
2020-12-16T07:11:04Z I! Loaded processors:
2020-12-16T07:11:04Z I! Loaded outputs: file influxdb_v2
2020-12-16T07:11:04Z I! Tags enabled: host=grafana
2020-12-16T07:11:04Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"grafana", Flush Interval:10s
2020-12-16T07:11:04Z D! [agent] Initializing plugins
2020-12-16T07:11:04Z D! [agent] Connecting outputs
2020-12-16T07:11:04Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2020-12-16T07:11:04Z D! [agent] Successfully connected to outputs.influxdb_v2
2020-12-16T07:11:04Z D! [agent] Attempting connection to [outputs.file]
2020-12-16T07:11:04Z D! [agent] Successfully connected to outputs.file
2020-12-16T07:11:04Z D! [agent] Starting service inputs
2020-12-16T07:11:04Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
2020-12-16T07:11:10Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2020-12-16T07:11:10Z D! [inputs.mqtt_consumer] Disconnected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Drehzahl value=4090 1608102664488479033
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3092 1608102664488842237
mqtt_consumer,host=grafana,topic=Heizen/WP/VorlaufTemp value=258 1608102669999739827
mqtt_consumer,host=grafana,topic=Heizen/WP/RuecklaufTemp value=237 1608102669999980930
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3137 1608102670000272333
2020-12-16T07:11:14Z D! [outputs.file] Wrote batch of 5 metrics in 34.3µs
2020-12-16T07:11:14Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:14Z D! [outputs.influxdb_v2] Wrote batch of 5 metrics in 74.27553ms
2020-12-16T07:11:14Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:20Z D! [inputs.mqtt_consumer] Connecting [tcp://10.100.124.24:1883]
2020-12-16T07:11:20Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/RuecklaufTemp value=239 1608102682755026952
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3025 1608102682755528258
2020-12-16T07:11:24Z D! [outputs.file] Wrote batch of 2 metrics in 22.901µs
2020-12-16T07:11:24Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:24Z D! [outputs.influxdb_v2] Wrote batch of 2 metrics in 89.732002ms
2020-12-16T07:11:24Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:30Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2020-12-16T07:11:30Z D! [inputs.mqtt_consumer] Disconnected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Drehzahl value=4076 1608102685698407635
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3092 1608102685699113843
mqtt_consumer,host=grafana,topic=Heizen/WP/VorlaufTemp value=260 1608102689766137676
mqtt_consumer,host=grafana,topic=Heizen/WP/RuecklaufTemp value=240 1608102689766744383
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3070 1608102689767300889
2020-12-16T07:11:34Z D! [outputs.file] Wrote batch of 5 metrics in 33.801µs
2020-12-16T07:11:34Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:34Z D! [outputs.influxdb_v2] Wrote batch of 5 metrics in 60.004371ms
2020-12-16T07:11:34Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:40Z D! [inputs.mqtt_consumer] Connecting [tcp://10.100.124.24:1883]
2020-12-16T07:11:40Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3204 1608102701976856760
2020-12-16T07:11:44Z D! [outputs.file] Wrote batch of 1 metrics in 14.7µs
2020-12-16T07:11:44Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:44Z D! [outputs.influxdb_v2] Wrote batch of 1 metrics in 44.7474ms
2020-12-16T07:11:44Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:50Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2020-12-16T07:11:50Z D! [inputs.mqtt_consumer] Disconnected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/VorlaufTemp value=262 1608102704913631256
mqtt_consumer,host=grafana,topic=Heizen/WP/RuecklaufTemp value=241 1608102704914184662
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3182 1608102704914302363
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3204 1608102708360162442
2020-12-16T07:11:54Z D! [outputs.file] Wrote batch of 4 metrics in 29.6µs
2020-12-16T07:11:54Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:11:54Z D! [outputs.influxdb_v2] Wrote batch of 4 metrics in 44.7834ms
2020-12-16T07:11:54Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:00Z D! [inputs.mqtt_consumer] Connecting [tcp://10.100.124.24:1883]
2020-12-16T07:12:00Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Drehzahl value=4136 1608102724079797044
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3048 1608102724079968846
2020-12-16T07:12:04Z D! [outputs.file] Wrote batch of 2 metrics in 24.3µs
2020-12-16T07:12:04Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:04Z D! [outputs.influxdb_v2] Wrote batch of 2 metrics in 44.540097ms
2020-12-16T07:12:04Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:10Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2020-12-16T07:12:10Z D! [inputs.mqtt_consumer] Disconnected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/VorlaufTemp value=264 1608102727532404883
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3092 1608102727532932389
2020-12-16T07:12:14Z D! [outputs.file] Wrote batch of 2 metrics in 21.5µs
2020-12-16T07:12:14Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:14Z D! [outputs.influxdb_v2] Wrote batch of 2 metrics in 44.662598ms
2020-12-16T07:12:14Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:20Z D! [inputs.mqtt_consumer] Connecting [tcp://10.100.124.24:1883]
2020-12-16T07:12:20Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Drehzahl value=4198 1608102740306309148
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3115 1608102740306440049
mqtt_consumer,host=grafana,topic=Energie/VH/Wirkleistung/Gesamt value=1024 1608102741169781284
mqtt_consumer,host=grafana,topic=Energie/VH/Wirkleistung/L1 value=606 1608102741199778718
mqtt_consumer,host=grafana,topic=Energie/VH/Wirkleistung/L2 value=59 1608102741231087768
mqtt_consumer,host=grafana,topic=Energie/VH/Wirkleistung/L3 value=358 1608102741260673498
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3070 1608102743238159564
mqtt_consumer,host=grafana,topic=Heizen/WP/VorlaufTemp value=266 1608102743238425367
2020-12-16T07:12:24Z D! [outputs.file] Wrote batch of 8 metrics in 41.2µs
2020-12-16T07:12:24Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:24Z D! [outputs.influxdb_v2] Wrote batch of 8 metrics in 53.916501ms
2020-12-16T07:12:24Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:30Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2020-12-16T07:12:30Z D! [inputs.mqtt_consumer] Disconnected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3115 1608102750266646887
2020-12-16T07:12:34Z D! [outputs.file] Wrote batch of 1 metrics in 29.2µs
2020-12-16T07:12:34Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:34Z D! [outputs.influxdb_v2] Wrote batch of 1 metrics in 44.401496ms
2020-12-16T07:12:34Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:40Z D! [inputs.mqtt_consumer] Connecting [tcp://10.100.124.24:1883]
2020-12-16T07:12:40Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3070 1608102762239971160
2020-12-16T07:12:44Z D! [outputs.file] Wrote batch of 1 metrics in 23.901µs
2020-12-16T07:12:44Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:44Z D! [outputs.influxdb_v2] Wrote batch of 1 metrics in 44.656298ms
2020-12-16T07:12:44Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:50Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2020-12-16T07:12:50Z D! [inputs.mqtt_consumer] Disconnected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/Drehzahl value=4254 1608102765704705507
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3182 1608102765705014310
mqtt_consumer,host=grafana,topic=Heizen/WP/VorlaufTemp value=268 1608102768596742663
2020-12-16T07:12:54Z D! [outputs.file] Wrote batch of 3 metrics in 43.901µs
2020-12-16T07:12:54Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:12:54Z D! [outputs.influxdb_v2] Wrote batch of 3 metrics in 44.569097ms
2020-12-16T07:12:54Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:13:00Z D! [inputs.mqtt_consumer] Connecting [tcp://10.100.124.24:1883]
2020-12-16T07:13:00Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/VorlaufTemp value=269 1608102781343525716
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3159 1608102781344924232
2020-12-16T07:13:04Z D! [outputs.file] Wrote batch of 2 metrics in 33µs
2020-12-16T07:13:04Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:13:04Z D! [outputs.influxdb_v2] Wrote batch of 2 metrics in 44.8859ms
2020-12-16T07:13:04Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:13:10Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2020-12-16T07:13:10Z D! [inputs.mqtt_consumer] Disconnected [tcp://10.100.124.24:1883]
mqtt_consumer,host=grafana,topic=Heizen/WP/RuecklaufTemp value=247 1608102784740520495
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3204 1608102784740755197
mqtt_consumer,host=grafana,topic=Heizen/WP/Durchfluss value=3159 1608102787665204704
2020-12-16T07:13:14Z D! [outputs.file] Wrote batch of 3 metrics in 35µs
2020-12-16T07:13:14Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:13:14Z D! [outputs.influxdb_v2] Wrote batch of 3 metrics in 44.296594ms
2020-12-16T07:13:14Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-12-16T07:13:20Z D! [inputs.mqtt_consumer] Connecting [tcp://10.100.124.24:1883]
2020-12-16T07:13:20Z I! [inputs.mqtt_consumer] Connected [tcp://10.100.124.24:1883]
5

Hello @matze1708,
You could use the execd processor plugin to convert your data as needed. telegraf/plugins/processors/execd at master · influxdata/telegraf · GitHub
Or perhaps
telegraf/plugins/processors/starlark at master · influxdata/telegraf · GitHub
Not sure about the last one.

6

Reading temperature values from MQTT works for me with:

## Data format to consume.
  ## Each data format has it's own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "value"
  data_type = "float"
7

Thanks for your answer.

Yes, i have float also in my mqtt input.
But, the broker send float with Comma not with a piont. Example 2,99 instead of 2.99

Is there a possible way, to handle this?

My workaround, i multipled the values in Broker with 100.

Thank you

1 Like