Shelly 1 PM MQTT issue

Hi,

while reading the MQTT topics from Shelly 2.5 and ShellyPlug usig Telegraf 1.22.4 is working fine for me, I am not able to get Telegraf to read it from the Shelly 1 PM MQTT topic.
My topic is “shellies/Shelly1PM-Office/status/switch:0” which make a JSON string available like this:

{
  "id": 0,
  "source": "init",
  "output": true,
  "apower": 254.2,
  "voltage": 241.8,
  "current": 1.562,
  "aenergy": {
    "total": 512.162,
    "by_minute": [
      1090.625,
      4010.086,
      4062.359
    ],
    "minute_ts": 1653487754
  },
  "temperature": {
    "tC": 56.9,
    "tF": 134.5
  }
}

My Telegraf MQTT config looks likes this

[[inputs.mqtt_consumer]]
  name_override = "shelly"
  servers = ["tcp://mqtt.home:1883"]
  connection_timeout = "60s"
  client_id = "telegraf-shelly1"
  topics = ["shellies/Shelly1PM-Office/status/switch:0"]
  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "+/+/+/+"
    measurement = "_/_/_/switch_id"
  
  data_format = "json_v2"
  [[inputs.mqtt_consumer.json_v2]]
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "apower"
        rename = "power"
        type = "float"
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "aenergy.total"
        rename = "energy"
        type = "float"
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "voltage"
        type = "float"
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "current"
        type = "float"

Although I have tried to comment out already bits and pieces, I get not output nor erros when running like this “telegraf.exe --config configs\mqtt_shelly1.conf --test”.
My output look likes this

D:\Monitoring\Telegraf>telegraf.exe --config configs\mqtt_shelly1.conf --test
2022-05-25T14:16:52Z I! Starting Telegraf 1.22.4
2022-05-25T14:16:52Z I! Loaded inputs: mqtt_consumer
2022-05-25T14:16:52Z I! Loaded aggregators:
2022-05-25T14:16:52Z I! Loaded processors: enum strings
2022-05-25T14:16:52Z W! Outputs are not used in testing mode!
2022-05-25T14:16:52Z I! Tags enabled: site=all
2022-05-25T14:16:52Z D! [agent] Initializing plugins
2022-05-25T14:16:52Z D! [agent] Starting service inputs
2022-05-25T14:16:52Z I! [inputs.mqtt_consumer] Connected [tcp://mqtt.home:1883]
2022-05-25T14:16:52Z D! [agent] Stopping service inputs
2022-05-25T14:16:52Z D! [inputs.mqtt_consumer] Disconnecting [tcp://mqtt.home:1883]
2022-05-25T14:16:52Z D! [inputs.mqtt_consumer] Disconnected [tcp://mqtt.home:1883]
2022-05-25T14:16:52Z D! [agent] Input channel closed
2022-05-25T14:16:52Z D! [agent] Processor channel closed
2022-05-25T14:16:52Z D! [agent] Processor channel closed
2022-05-25T14:16:52Z D! [agent] Stopped Successfully

Anyone any idea what I am doing wrong here?

Hi @mengrie,
Can you try running:

telegraf.exe --config configs\mqtt_shelly1.conf --debug

I am not seeing any issues jumping out at me yet.

D:\Monitoring\Telegraf>telegraf.exe --config configs\mqtt_shelly1.conf --debug
2022-05-26T14:34:06Z I! Starting Telegraf 1.22.4
2022-05-26T14:34:06Z I! Loaded inputs: mqtt_consumer
2022-05-26T14:34:06Z I! Loaded aggregators:
2022-05-26T14:34:06Z I! Loaded processors: enum strings
2022-05-26T14:34:06Z I! Loaded outputs: influxdb
2022-05-26T14:34:06Z I! Tags enabled: site=all
2022-05-26T14:34:06Z I! [agent] Config: Interval:5m0s, Quiet:false, Hostname:"", Flush Interval:5m0s
2022-05-26T14:34:06Z D! [agent] Initializing plugins
2022-05-26T14:34:06Z D! [agent] Connecting outputs
2022-05-26T14:34:06Z D! [agent] Attempting connection to [outputs.influxdb]
2022-05-26T14:34:06Z D! [agent] Successfully connected to outputs.influxdb
2022-05-26T14:34:06Z D! [agent] Starting service inputs
2022-05-26T14:34:06Z I! [inputs.mqtt_consumer] Connected [tcp://mqtt.home:1883]
2022-05-26T14:34:09Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:9: “{“id”:0, “source”:“init”, “output”:true, “apower”:255.4, “voltage”:236.2, “current”:1.550, “aenergy”:{“total”:5078.429,“by_minute”:[743.965,4445.553,4497.435],“minute_ts”:1653575648},“temperature”:{“tC”:57.7, “tF”:135.8}}”

Hmm!? OK. There is an error but I have no clue how to fix it

hmm, can you send me your full Telegraf config, please? I cannot see why it’s looking for a tag as based on the config you have shown me you haven’t defined one. Could you also humour me and comment out the topic parsing and see if that changes the result?

Euh ??? “Could you also humour me” ???

Here is the full config

#--------------------------------------------------------------------------------------------------
# Global tags can be specified here in key="value" format.
#--------------------------------------------------------------------------------------------------
[global_tags]
  site = "all"

#--------------------------------------------------------------------------------------------------
# Configuration for telegraf agent
#--------------------------------------------------------------------------------------------------
[agent]
  ## Default data collection interval for all inputs
  ## we do a check every 5 minutes
  interval = "300s"
  ## Rounds collection interval to 'interval'
  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
  round_interval = true

  ## Telegraf will send metrics to outputs in batches of at most
  ## metric_batch_size metrics.
  ## This controls the size of writes that Telegraf sends to output plugins.
  metric_batch_size = 1000

  ## Maximum number of unwritten metrics per output.
  ## Increasing this value allows for longer periods of output downtime without
  ## dropping metrics at the cost of higher maximum memory usage.
  metric_buffer_limit = 10000

  ## Collection jitter is used to jitter the collection by a random amount.
  ## Each plugin will sleep for a random time within jitter before collecting.
  ## This can be used to avoid many plugins querying things like sysfs at the
  ## same time, which can have a measurable effect on the system.
  collection_jitter = "60s"

  ## Default flushing interval for all outputs.
  ## Maximum flush_interval will be flush_interval + flush_jitter
  flush_interval = "300s"
  ## Jitter the flush interval by a random amount. This is primarily to avoid
  ## large write spikes for users running a large number of telegraf instances.
  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
  flush_jitter = "60s"

  ## By default or when set to "0s", precision will be set to the same
  ## timestamp order as the collection interval, with the maximum being 1s.
  ##   ie, when interval = "300s", precision will be "1s"
  ##       when interval = "250ms", precision will be "1ms"
  ## Precision will NOT be used for service inputs.
  ## It is up to each individual service input to set the timestamp at the appropriate precision.
  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
  precision = "0s"

  ## Log at debug level.
  debug = true
  ## Log only error level messages.
  quiet = false

  ## Log target controls the destination for logs and can be one of "file",
  ## "stderr" or, on Windows, "eventlog".  When set to "file", the output file
  ## is determined by the "logfile" setting.
  # logtarget = "file"

  ## Name of the file to be logged to when using the "file" logtarget.  If set to
  ## the empty string then logs are written to stderr.
  # logfile = ""

  ## The logfile will be rotated after the time interval specified.  When set
  ## to 0 no time based rotation is performed.  Logs are rotated only when
  ## written to, if there is no log activity rotation may be delayed.
  # logfile_rotation_interval = "0d"

  ## The logfile will be rotated when it becomes larger than the specified
  ## size.  When set to 0 no size based rotation is performed.
  # logfile_rotation_max_size = "0MB"

  ## Maximum number of rotated archives to keep, any older logs are deleted.
  ## If set to -1, no archives are removed.
  # logfile_rotation_max_archives = 5

  ## Override default hostname, if empty use os.Hostname()
  hostname = ""
  ## If set to true, do no set the "host" tag in the telegraf agent.
  omit_hostname = true

#--------------------------------------------------------------------------------------------------
# Processors
#--------------------------------------------------------------------------------------------------
[[processors.strings]]
  [[processors.strings.lowercase]]
    tag = "host"

[[processors.enum]]
  [[processors.enum.mapping]]
    ## Name of the tag to map
    tag = "url"

    ## Destination tag or field to be used for the mapped value.  By default the
    ## source tag or field is used, overwriting the original value.
    dest = "url_name"

    ## Default value to be used for all values not contained in the mapping
    ## table.  When unset and no match is found, the original field will remain
    ## unmodified and the destination tag or field will not be created.
    default = ""

#--------------------------------------------------------------------------------------------------
# Output
#--------------------------------------------------------------------------------------------------
[[outputs.influxdb]]
  urls = ["http://localhost:8086"]
  username = "admin"
  password = "*****"
  database = "mqtt"

#--------------------------------------------------------------------------------------------------
# Inputs
#--------------------------------------------------------------------------------------------------

[[inputs.mqtt_consumer]]

  name_override = "shelly"

  ## MQTT broker URLs to be used. The format should be scheme://host:port,
  ## schema can be tcp, ssl, or ws.
  servers = ["tcp://mqtt.home:1883"]

  ## Connection timeout for initial connection in seconds
  connection_timeout = "60s"

  ## If unset, a random client ID will be generated.
  client_id = "telegraf-shelly1"

  topics = ["shellies/Shelly1PM-Office/status/switch:0"]
  [[inputs.mqtt_consumer.topic_parsing]]
    topic = "+/+/+/+"
    measurement = "_/_/_/switch_id"
  
  data_format = "json_v2"
  [[inputs.mqtt_consumer.json_v2]]
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "apower"
        rename = "power"
        type = "float"
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "aenergy.total"
        rename = "energy"
        type = "float"
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "voltage"
        type = "float"
    [[inputs.mqtt_consumer.json_v2.field]]
        path = "current"
        type = "float"
  
# telegraf.exe --config configs\mqtt_shelly1.conf --test
# telegraf.exe --config configs\mqtt_shelly1.conf --debug

when commenting out topis parsing not much is showing

D:\Monitoring\Telegraf>telegraf.exe --config configs\mqtt_shelly1.conf --debug
2022-05-26T15:15:28Z I! Starting Telegraf 1.22.4
2022-05-26T15:15:28Z I! Loaded inputs: mqtt_consumer
2022-05-26T15:15:28Z I! Loaded aggregators:
2022-05-26T15:15:28Z I! Loaded processors: enum strings
2022-05-26T15:15:28Z I! Loaded outputs: influxdb
2022-05-26T15:15:28Z I! Tags enabled: site=all
2022-05-26T15:15:28Z I! [agent] Config: Interval:5m0s, Quiet:false, Hostname:"", Flush Interval:5m0s
2022-05-26T15:15:28Z D! [agent] Initializing plugins
2022-05-26T15:15:28Z D! [agent] Connecting outputs
2022-05-26T15:15:28Z D! [agent] Attempting connection to [outputs.influxdb]
2022-05-26T15:15:28Z D! [agent] Successfully connected to outputs.influxdb
2022-05-26T15:15:28Z D! [agent] Starting service inputs
2022-05-26T15:15:28Z I! [inputs.mqtt_consumer] Connected [tcp://mqtt.home:1883]
2022-05-26T15:20:24Z E! [inputs.mqtt_consumer] Error in plugin: connection lost: EOF
2022-05-26T15:20:24Z D! [inputs.mqtt_consumer] Disconnected [tcp://mqtt.home:1883]
2022-05-26T15:20:53Z D! [outputs.influxdb] Wrote batch of 22 metrics in 32.2144ms
2022-05-26T15:20:53Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2022-05-26T15:20:59Z D! [inputs.mqtt_consumer] Connecting [tcp://mqtt.home:1883]
2022-05-26T15:20:59Z I! [inputs.mqtt_consumer] Connected [tcp://mqtt.home:1883]

Haha!!! commenting out topic parsing does wirtie data to InfluxDB

time	current	energy	power	site	topic	value	voltage
26/05/2022 15:18:59	1234	5249399	205	all	shellies/Shelly1PM-Office/status/switch:0		238.8
26/05/2022 15:19:11	1229	5250095	221.8	all	shellies/Shelly1PM-Office/status/switch:0		238.5
26/05/2022 15:19:13	1246	5250.21	207.7	all	shellies/Shelly1PM-Office/status/switch:0		238.5
26/05/2022 15:19:59	1269	5252846	209.4	all	shellies/Shelly1PM-Office/status/switch:0		237.9
26/05/2022 15:20:17	1322	5253905	220.9	all	shellies/Shelly1PM-Office/status/switch:0		237.8
26/05/2022 15:20:17	1322	5253905	220.9	all	shellies/Shelly1PM-Office/status/switch:0		237.8
26/05/2022 15:21:01	1245	5256486	206.8	all	shellies/Shelly1PM-Office/status/switch:0		237.5
26/05/2022 15:21:57	1.31	5259701	222.7	all	shellies/Shelly1PM-Office/status/switch:0		238.1
26/05/2022 15:21:57	1.31	5259701	222.7	all	shellies/Shelly1PM-Office/status/switch:0		238.1
26/05/2022 15:21:59	1.31	5259816	208.5	all	shellies/Shelly1PM-Office/status/switch:0		238.1
26/05/2022 15:21:59	1.31	5259816	208.5	all	shellies/Shelly1PM-Office/status/switch:0		238.1
26/05/2022 15:22:05	1.24	5260167	206.8	all	shellies/Shelly1PM-Office/status/switch:0		238.3
26/05/2022 15:22:41	1321	5262274	220.9	all	shellies/Shelly1PM-Office/status/switch:0		238.6
26/05/2022 15:22:41	1321	5262274	220.9	all	shellies/Shelly1PM-Office/status/switch:0		238.6
26/05/2022 15:22:45	1421	5262.53	243.9	all	shellies/Shelly1PM-Office/status/switch:0		238.8
26/05/2022 15:22:45	1421	5262.53	243.9	all	shellies/Shelly1PM-Office/status/switch:0		238.8
26/05/2022 15:22:47	1421	5262654	223.6	all	shellies/Shelly1PM-Office/status/switch:0		238.9

Sorry @mengrie,
I had a google and bad use of the saying “can you humour me” :slight_smile:. I meant no offence. I was just asking you to do something silly so meant with that context :smiley: . I am now glad you have data but lets see if we can get your topic parsing working :slight_smile:

No offence taken :wink: (Because my native language is Dutch, I wondering if it meant something special …)

It would be great to have the topic parsing working. I sure wanna go for it.
But if not, I want stop eating because of this :smile:

Hi,
this topic is one year old, but I ran into a similar issue.

I always got the error

plugin inputs.mqtt_consumer: line 12: configuration specified the fields ["json_v2"], but they weren't used

I found out that placing the topic parsing section

[[inputs.mqtt_consumer.topic_parsing]]

after the Json_v2 section

 data_format = "json_v2"
  [[inputs.mqtt_consumer.json_v2]]

solved the issue.

This way json parsing and topic parsing works.

This happened with telegraf version Telegraf 1.27.3 (git: HEAD@afcf0133)