Help getting MQTT feed from Solar Assistant via MQTT > Talegraf to Influxdb

Telegraf
, ,
1

Hi all, I’m brand new to telegraf and MQTT and struggling to get the data from my new Solar Assistant setup into my InfluxDB V2.

Setup:
1 Raspi3b running Solar Assistant V2023-02-28

  1. Raspi4b running:

InfluxDB V2.6.1
Telegraf V1.26.0
MosquittoV2.0.11
Pi’s running Raspberry Pi OS Lite (64-bit)

I have MQTT set up on the Solar Assistant and can subscribe to it using the solar assistant IP from the console and get the following output This is a subset.

solar_assistant/total/battery_power/state 0
solar_assistant/total/battery_state_of_charge/state 100
solar_assistant/total/battery_temperature/state 16.0
solar_assistant/total/bus_voltage/state 855.3
solar_assistant/inverter_1/free_mode_charge_rate/state 100
solar_assistant/inverter_1/max_charge_current/state 60.0
solar_assistant/inverter_1/battery_absorption_charge_voltage/state 58.0
solar_assistant/inverter_1/grid_frequency/state 50.0
solar_assistant/inverter_1/pv_current_1/state 2.3
solar_assistant/inverter_1/load_first_stop_discharge/state 5
solar_assistant/inverter_1/pv_power/state 1823
solar_assistant/inverter_1/battery_voltage/state 53.2
solar_assistant/inverter_1/grid_first_stop_discharge/state 5
solar_assistant/inverter_1/pv_current_2/state 3.6
solar_assistant/inverter_1/temperature/state 33.9
solar_assistant/inverter_1/battery_float_charge_voltage/state 57.5
solar_assistant/inverter_1/load_first_stop_charge/state 100
solar_assistant/inverter_1/battery_current/state 0.0
solar_assistant/inverter_1/battery_first_grid_charge/state Disabled
solar_assistant/inverter_1/output_source_priority/state Load first
solar_assistant/inverter_1/grid_first_discharge_rate/state 100
solar_assistant/inverter_1/grid_power/state -1550
solar_assistant/inverter_1/pv_voltage_1/state 301.2
solar_assistant/inverter_1/pv_voltage_2/state 302.9
solar_assistant/inverter_1/pv_power_1/state 714
solar_assistant/inverter_1/battery_first_charge_rate/state 100
solar_assistant/inverter_1/device_mode/state Hybrid
solar_assistant/inverter_1/free_mode_discharge_rate/state 100
solar_assistant/inverter_1/battery_first_stop_charge/state 100
solar_assistant/inverter_1/grid_voltage/state 240.0
solar_assistant/inverter_1/load_power/state 200
solar_assistant/inverter_1/pv_power_2/state 1109
solar_assistant/total/battery_power/state 0
solar_assistant/total/battery_state_of_charge/state 100
solar_assistant/total/battery_temperature/state 16.0
solar_assistant/total/bus_voltage/state 855.3
solar_assistant/inverter_1/free_mode_charge_rate/state 100
solar_assistant/inverter_1/max_charge_current/state 60.0
solar_assistant/inverter_1/battery_absorption_charge_voltage/state 58.0
solar_assistant/inverter_1/grid_frequency/state 50.0
solar_assistant/inverter_1/pv_current_1/state 2.3
solar_assistant/inverter_1/load_first_stop_discharge/state 5
solar_assistant/inverter_1/pv_power/state 1823
solar_assistant/inverter_1/battery_voltage/state 53.2
solar_assistant/inverter_1/grid_first_stop_discharge/state 5
solar_assistant/inverter_1/pv_current_2/state 3.6
solar_assistant/inverter_1/temperature/state 33.9
solar_assistant/inverter_1/battery_float_charge_voltage/state 57.5
solar_assistant/inverter_1/load_first_stop_charge/state 100
solar_assistant/inverter_1/battery_current/state 0.0
solar_assistant/inverter_1/battery_first_grid_charge/state Disabled
solar_assistant/inverter_1/output_source_priority/state Load first
solar_assistant/inverter_1/grid_first_discharge_rate/state 100
solar_assistant/inverter_1/grid_power/state -1550
solar_assistant/inverter_1/pv_voltage_1/state 301.2
solar_assistant/inverter_1/pv_voltage_2/state 302.9
solar_assistant/inverter_1/pv_power_1/state 714
solar_assistant/inverter_1/battery_first_charge_rate/state 100
solar_assistant/inverter_1/device_mode/state Hybrid
solar_assistant/inverter_1/free_mode_discharge_rate/state 100
solar_assistant/inverter_1/battery_first_stop_charge/state 100
solar_assistant/inverter_1/grid_voltage/state 240.0
solar_assistant/inverter_1/load_power/state 200
solar_assistant/inverter_1/pv_power_2/state 1109
solar_assistant/total/battery_power/state 0
solar_assistant/total/battery_state_of_charge/state 100
solar_assistant/total/battery_temperature/state 16.0
solar_assistant/total/bus_voltage/state 855.3

I have the following in /etc/mosquitto/conf.d/solar-assistant.conf

connection SolarAssistant
#remote_username solar-assistant
#remote_password solar123
address xxx.xxx.xxx.xxx:1883
topic # in 0
topic solar_assistant/# out 0

I can subscribe to it using the localhost address and still get the same output.

So far, so good I think.

The difficulty I am having is getting telegraf to subscribe to the same feed and put the data into influxdb.

I have the following in my telegraf.conf file…

[agent]
omit_hostname = true

[[inputs.mqtt_consumer]]
servers = ["tcp://localhost:1883"]
topics = [ "solar_assistant" ]
data_format = "influx"
topic_tag = ""

[[outputs.influxdb_v2]]
urls = ["http://localhost:8086"]
token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
organization = "xxxxxx"
namepass = ["solar_assistant"]
bucket = "xxxxx"

This seems to start up OK from the following terminal OP

$ sudo systemctl start telegraf
$ sudo systemctl status telegraf
● telegraf.service - Telegraf
     Loaded: loaded (/lib/systemd/system/telegraf.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-03-27 16:19:55 BST; 4s ago
       Docs: https://github.com/influxdata/telegraf
   Main PID: 23909 (telegraf)
      Tasks: 9 (limit: 8755)
        CPU: 266ms
     CGroup: /system.slice/telegraf.service
             └─23909 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d

Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! Loaded inputs: mqtt_consumer
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! Loaded aggregators:
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! Loaded processors:
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! Loaded secretstores:
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! Loaded outputs: influxdb_v2
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! Tags enabled:
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:10s
Mar 27 16:19:55 PiEnergyMon systemd[1]: Started Telegraf.
Mar 27 16:19:55 PiEnergyMon telegraf[23909]: 2023-03-27T15:19:55Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]

But I’m not getting the data into Influxdb.

I know my token etc is OK.
As I say, I’m brand new to this and at a loss as to where to go from here. I’m guessing once I get it running I’m going to have to parse the data to get it in in some sensible form but I want to see what it produces first of all. Plus I know I’ll need to think about security a bit more once it’s running.
Any help appreciated.

2

This

  topics = [ "solar_assistant" ]

only subscribes to exactly the solar_assistant topic, not to all subtopics… You have to use

  topics = [ "solar_assistant/#" ]

See e.g. this documentation to understand MQTT wildcards…

3

Thanks @srebhan that explains a lot.
The only problem is I can’t start telegraf. It fails with parse errors in inputs.mqtt_consumer…

$ sudo systemctl status telegraf
● telegraf.service - Telegraf
     Loaded: loaded (/lib/systemd/system/telegraf.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-03-28 09:54:30 BST; 9min ago
       Docs: https://github.com/influxdata/telegraf
   Main PID: 35612 (telegraf)
      Tasks: 10 (limit: 8755)
        CPU: 14.468s
     CGroup: /system.slice/telegraf.service
             └─35612 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d

Mar 28 10:04:16 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:16Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "315.4"
Mar 28 10:04:16 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:16Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "305.1"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "328"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "100"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:7: "Hybrid"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "100"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "100"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:6: "239.7"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "330"
Mar 28 10:04:17 PiEnergyMon telegraf[35612]: 2023-03-28T09:04:17Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "308"
clive@PiEnergyMon:~ $

If I run this from the command line I get the data feed OK.
mosquitto_sub -h localhost -p 1883 -v -t 'solar_assistant/#'

My current telegraf.conf file contains:

[agent]
omit_hostname = true

[[inputs.mqtt_consumer]]
servers = [ "tcp://localhost:1883" ]
topics = [ "solar_assistant/#" ]
data_format = "influx"
topic_tag = ""

[[outputs.influxdb_v2]]
urls = ["http://localhost:8086"]
token = "IDZTADcyUyYAkze8CnwDpUy0PJhKSod3xay_m08q50oJGNsQWX-Xz4DFt7f1UV6iSJAmrBK3tSDfQCOqpjh0bQ=="
organization = "reynolds"
namepass = [""]
bucket = "growattdb"

My /etc/mosquitto/conf.d/solar-assistant.conf hasn’t changed but I assume ot’s ok because I can see it coming out of the bridge on the command line.

connection SolarAssistant
#remote_username solar-assistant
#remote_password solar123
address **.***.***.***:1883
topic # in 0
topic solar_assistant/# out 0

I’m assuming I have something wrong syntax wise in the input section of my telegraf.conf but I’m blowed if I can see what’s wrong.

Edit: Just noticed: It’s the same if I go back to topics = [ “solar_assistant” ] or if I use topics = [ “#” ]

Sorry, I also notice that telegraf is in fact running but I am getting the parse errors on the input plugin when I use topics = [ “solar_assistant/#” ]
When I run it with just "solar_assitant it runs with no errors.
So I’m guessing this means there is a problem with the data coming in from mqtt?

5

In My telegraf.conf file I have
data_format = “influx”

Well but your data isn’t Influx line protocol and that’s why you get the error! Try to use the value parser with

  data_format = "value"
  data_type = "float"

for the float values. You likely need to subscribe to with multiple MQTT consumers and use string or integer types for the corresponding topics.

6

Sorry to ask so many (probably) dumb questions.
Are you saying that, in this line for instance…

that the topic it pv_power with the value at the end in state 1823?

7

Before I jump in and try this: Am I reading it right?
Will the name_override declaration rename each _measurement to that value?
Therefore I need a separate [[inputs.mqtt_consumer]] section for each metric and I can then name it something meaningful but simple?

Something along these lines…

[[inputs.mqtt_consumer]]
[[inputs.exec]]
  ## Commands array
  commands = ["cat /proc/sys/kernel/random/entropy_avail"]

  ## override the default metric name of "exec"
  name_override = "my name for metric"

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ##   https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "value"
  data_type = "integer" # required

servers = [ "tcp://localhost:1883" ]
#Integer Values
topics = [ "solar_assistant/inverter_1/pv_power_2" ]
#data_format = "influx"
#topic_tag = ""

Repeating this block for each metric changing the name_override and data_type in each case as appropriate?
Do I still need the data_format and topic_tag entries at the bottom?
Is the way I have laid it out correct/does it matter what order I put things in?

Many thanks for all your help by the way. much appreciated.

8

Sorry to keep posting questions but I am struggling to find the info I need online.

I have got a .conf file that runs now but still not getting anything into my bucket.

I’d just like an expert to cast an eye over my .conf file to check I am not going off on the wrong track altogether and to see where I might be going wrong.

My areas of confusion are:

  1. Do I need to add/change anything in the Commands Array
  2. Am I right in thinking the name_override will set the _measurement name in my bucket?
  3. If 2 is true then I assume I will need a separate metric_name.conf file in /etc/telegraf/telegraf.d for each metric?
  4. Am I right in matching the data_format in both [[inputs.exec]] and [[inputs.mqtt_consumer]]?

I hope I am moving in the right direction and have just missed something stupid. I am very much an ageing beginner at all this so any help is greatly appreciated.

[agent]
omit_hostname = true

[[inputs.exec]]
  ## Commands array
  commands = ["cat /proc/sys/kernel/random/entropy_avail"]

  ## override the default metric name of "exec"
  name_override = "pv_power"

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ##   https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "value"
  data_type = "float" # required

[[inputs.mqtt_consumer]]
servers = [ "tcp://localhost:1883" ]
#Integer Values
topics = [ "solar_assistant/inverter_1/pv_power" ]

data_format = "value"
topic_tag = ""

[[outputs.influxdb_v2]]
urls = ["http://localhost:8086"]
token = "*************************************************************************************"
organization = "********"
namepass = [""]
bucket = "growattdb"

The terminal OP for sytemctl1 status telegraf is …

$ systemctl status telegraf
● telegraf.service - Telegraf
     Loaded: loaded (/lib/systemd/system/telegraf.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-03-29 10:44:02 BST; 5s ago
       Docs: https://github.com/influxdata/telegraf
   Main PID: 52829 (telegraf)
      Tasks: 9 (limit: 8755)
        CPU: 242ms
     CGroup: /system.slice/telegraf.service
             └─52829 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d

Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! Loaded inputs: exec mqtt_consumer
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! Loaded aggregators:
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! Loaded processors:
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! Loaded secretstores:
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! Loaded outputs: influxdb_v2
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! Tags enabled:
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:10s
Mar 29 10:44:02 PiEnergyMon systemd[1]: Started Telegraf.
Mar 29 10:44:02 PiEnergyMon telegraf[52829]: 2023-03-29T09:44:02Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
9

OK, with a lot of experimenting and googling I now have it working for one of the metrics being posted.
My telegraf.conf now looks like this…

[agent]
omit_hostname = true

[[inputs.mqtt_consumer]]
        servers = [ "tcp://localhost:1883" ]
        topics = [ "solar_assistant/inverter_1/pv_power/state" ]
        data_format = "value"
        data_type = "integer"
        topic_tag = ""

[[inputs.mqtt_consumer.topic_parsing]]
        topic = "solar_assistant/inverter_1/pv_power/state"
        measurement = "_/measurement/_/_"
        tags = "_/_/sensor_id/_"


[[outputs.influxdb_v2]]
        urls = ["http://localhost:8086"]
        token = "****************************************************************"
        organization = "***************"
        bucket = "******"

I am getting data into infuxdb now and it looks good.

My next job is to add all the rest of my metrics and I could do with some advice on the best way forward.

These are the metrics I need to add.

solar_assistant/inverter_1/pv_power/state 1162
solar_assistant/inverter_1/battery_voltage/state 53.2
solar_assistant/inverter_1/grid_first_stop_discharge/state 5
solar_assistant/inverter_1/pv_current_2/state 2.1
solar_assistant/inverter_1/temperature/state 36.5
solar_assistant/inverter_1/battery_float_charge_voltage/state 57.5
solar_assistant/inverter_1/load_first_stop_charge/state 100
solar_assistant/inverter_1/battery_current/state 0.0
solar_assistant/inverter_1/battery_first_grid_charge/state Disabled
solar_assistant/inverter_1/output_source_priority/state Load first
solar_assistant/inverter_1/grid_first_discharge_rate/state 100
solar_assistant/inverter_1/grid_power/state -920
solar_assistant/inverter_1/pv_voltage_1/state 303.8
solar_assistant/inverter_1/pv_voltage_2/state 306.5
solar_assistant/inverter_1/pv_power_1/state 499
solar_assistant/inverter_1/battery_first_charge_rate/state 100
solar_assistant/inverter_1/device_mode/state Hybrid
solar_assistant/inverter_1/free_mode_discharge_rate/state 100
solar_assistant/inverter_1/battery_first_stop_charge/state 100
solar_assistant/inverter_1/grid_voltage/state 240.0
solar_assistant/inverter_1/load_power/state 240
solar_assistant/inverter_1/pv_power_2/state 663
solar_assistant/total/battery_power/state 0
solar_assistant/total/battery_state_of_charge/state 99
solar_assistant/total/battery_temperature/state 19.0
solar_assistant/total/bus_voltage/state 855.2
solar_assistant/inverter_1/free_mode_charge_rate/state 100
solar_assistant/inverter_1/max_charge_current/state 60.0
solar_assistant/inverter_1/battery_absorption_charge_voltage/state 58.0
solar_assistant/inverter_1/grid_frequency/state 49.9
solar_assistant/inverter_1/pv_current_1/state 1.6
solar_assistant/inverter_1/load_first_stop_discharge/state 5

There are a mix of integer and float results.
So:
Do I need to move my telegraf.conf as it is to /etc/telegraf/telegraf.d/metricname.conf and create a seperate file for each metric?
If I do what (if anything) do I need in my telegraf.conf?
Am I right is thinking anything in telegraf.d will override what is in telegraf.conf?
Or, can I just add multiple [[inputs.mqtt_consumer]] and [[inputs.mqtt_consumer.topic_parsing]] sections in the same .config file?
Any advice apprfeciated.

10

I seem to have it working, at least for the integer values without using the value parser you refer to.
This works for several of the integer metrics…

For example
solar_assistant/inverter_1/pv_power/state 1162

[[inputs.mqtt_consumer]]
        servers = [ "tcp://localhost:1883" ]
        topics = [ "solar_assistant/inverter_1/pv_power/state" ]
        data_format = "value"
        data_type = "integer"
        topic_tag = ""

[[inputs.mqtt_consumer.topic_parsing]]
        topic = "solar_assistant/inverter_1/pv_power/state"
        measurement = "_/measurement/_/_"
        tags = "_/_/sensor_id/_"

However for the integer metrics I get the _measurement and sensor_id but no value using the same format as above…
solar_assistant/inverter_1/grid_voltage/state 240.0

[[inputs.mqtt_consumer]]
        servers = [ "tcp://localhost:1883" ]
        topics = [ "solar_assistant/total/grid_voltage/state" ]
        data_format = "value"
        data_type = "integer"
        topic_tag = ""

[[inputs.mqtt_consumer.topic_parsing]]
        topic = "solar_assistant/total/grid_voltage/state"
        measurement = "_/measurement/_/_"
        tags = "_/_/sensor_id/_"

Is this why I need the value parser plugin? if so how and where do I implement it?
I did try it before but I couldn’t get telegraf to start wherever I put the [[inputs.exec]] code.
I thought I had this but it seems my knowledge is still very much lacking so any help greatly appreciated.

11

Exactly. The grid_voltage topic has a floating point value which cannot be interpreted as integer! If you know which fields are of which type you could take two approaches

  1. Have multiple mqtt_consumer plugins each subscribing to the topics of the same type and with different settings for data_type corresponding to the topics’ data-type
  2. Have one mqtt_consumer plugin with data_type = "string", use pivot processor to convert the sensor_id tag to your field name and finally use the converter processor to convert the fields to the correct type.

If you do not know the topic/data-type mapping a-priori you can only use the starlark processor and try to automatically determine the type…

1 Like
12

I’m sorry @srebhan , I must have been stoned when I put up my last post.
I am using you first suggestion at the moment.

  1. Have multiple mqtt_consumer plugins each subscribing to the topics of the same type and with different settings for data_type corresponding to the topics’ data-type
    I can indeed see what data type each field is.
    The integer data types are working fine.
    The float types, with data_type = “float” doesn’t work. I get the _measurement and sensor_id but no values posted.

So, for the topic item…
solar_assistant/inverter_1/grid_voltage/state 240.0
the corresponding section in my .conf that isn’t working is …

[[inputs.mqtt_consumer]]
        servers = [ "tcp://localhost:1883" ]
        topics = [ "solar_assistant/total/grid_voltage/state" ]
        data_format = "value"
        data_type = "float"
        topic_tag = ""

[[inputs.mqtt_consumer.topic_parsing]]
        topic = "solar_assistant/total/grid_voltage/state"
        measurement = "_/measurement/_/_"
        tags = "_/_/sensor_id/_"

So, if I’m reading you suggestion right this should work? which is what I figured and was working to until the float types didn’t work. This led to my last post and the question as to why the float types don’t work in my case.
I’ll take a look at the possibility of using your second suggestion using the pivot and converter processors, but can anyone see why the code above doesn’t work?

13

@cliver yes this should work if you can guarantee that only valid floating point values are sent to that topic i.e there is no offline value of similar. To find out what is going on, it is necessary to know the exact data (i.e. the value in that topic) for which the above is failing and the exact error message.

14

@srebhan , thanks for confirming my logic is at least on the right track.

Here is a page of output when I subscribe to the above topic via the terminal…

solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.9
solar_assistant/inverter_1/grid_voltage/state 238.9
solar_assistant/inverter_1/grid_voltage/state 238.9
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.0
solar_assistant/inverter_1/grid_voltage/state 239.0
solar_assistant/inverter_1/grid_voltage/state 239.0
solar_assistant/inverter_1/grid_voltage/state 239.0
solar_assistant/inverter_1/grid_voltage/state 239.0
solar_assistant/inverter_1/grid_voltage/state 239.0
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 238.8
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.3
solar_assistant/inverter_1/grid_voltage/state 239.3
solar_assistant/inverter_1/grid_voltage/state 239.3
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.6
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.4
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.2
solar_assistant/inverter_1/grid_voltage/state 239.1
solar_assistant/inverter_1/grid_voltage/state 239.1

I take this to show that the output is indeed float. An I right to assume this at this point?

When I run telegraf I don’t see any errors, it seems to connect to the inputs OK but I get no output ( I have tried setting up an output to file but the file is empty).

The output from telegraf --config /etc/telegraf/telegraf.conf --test is…

$ telegraf --config /etc/telegraf/telegraf.conf --test
2023-04-04T11:51:02Z I! Loading config file: /etc/telegraf/telegraf.conf
2023-04-04T11:51:02Z I! Starting Telegraf 1.26.0
2023-04-04T11:51:02Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-04-04T11:51:02Z I! Loaded inputs: mqtt_consumer (6x)
2023-04-04T11:51:02Z I! Loaded aggregators:
2023-04-04T11:51:02Z I! Loaded processors:
2023-04-04T11:51:02Z I! Loaded secretstores:
2023-04-04T11:51:02Z W! Outputs are not used in testing mode!
2023-04-04T11:51:02Z I! Tags enabled:
2023-04-04T11:51:02Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:51:02Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:51:02Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:51:02Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:51:02Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:51:02Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]

Running it with --once …

 $ telegraf --config /etc/telegraf/telegraf.conf --once
2023-04-04T11:57:18Z I! Loading config file: /etc/telegraf/telegraf.conf
2023-04-04T11:57:18Z I! Starting Telegraf 1.26.0
2023-04-04T11:57:18Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-04-04T11:57:18Z I! Loaded inputs: mqtt_consumer (6x)
2023-04-04T11:57:18Z I! Loaded aggregators:
2023-04-04T11:57:18Z I! Loaded processors:
2023-04-04T11:57:18Z I! Loaded secretstores:
2023-04-04T11:57:18Z I! Loaded outputs: file
2023-04-04T11:57:18Z I! Tags enabled:
2023-04-04T11:57:18Z D! [agent] Initializing plugins
2023-04-04T11:57:18Z D! [agent] Connecting outputs
2023-04-04T11:57:18Z D! [agent] Attempting connection to [outputs.file]
2023-04-04T11:57:18Z D! [agent] Successfully connected to outputs.file
2023-04-04T11:57:18Z D! [agent] Starting service inputs
2023-04-04T11:57:18Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:57:18Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:57:18Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:57:18Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:57:18Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:57:18Z I! [inputs.mqtt_consumer] Connected [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [agent] Stopping service inputs
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnecting [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnected [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnecting [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnected [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnecting [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnected [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnecting [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnected [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnecting [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnected [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnecting [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [inputs.mqtt_consumer] Disconnected [tcp://localhost:1883]
2023-04-04T11:57:18Z D! [agent] Input channel closed
2023-04-04T11:57:18Z I! [agent] Hang on, flushing any cached metrics before shutdown
2023-04-04T11:57:18Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2023-04-04T11:57:18Z I! [agent] Stopping running outputs
2023-04-04T11:57:18Z D! [agent] Stopped Successfully

If I run

telegraf --config /etc/telegraf/telegraf.conf

with the output to file and debugging=true
I see no errors reported and all the integer inputs are fed to the OP file but not the float one.

 mqttout.txt
  GNU nano 5.4                                                                                                                                             mqttout.txt
total,sensor_id=battery_power value=0i 1680610197781447288
total,sensor_id=battery_state_of_charge value=100i 1680610197782847009
inverter_1,sensor_id=pv_power value=4081i 1680610197821136593
inverter_1,sensor_id=grid_power value=-3690i 1680610197842993958
inverter_1,sensor_id=load_power value=380i 1680610197873027581
total,sensor_id=battery_power value=0i 1680610198388013520
total,sensor_id=battery_state_of_charge value=100i 1680610198392281183
inverter_1,sensor_id=pv_power value=4073i 1680610198409724535
inverter_1,sensor_id=grid_power value=-3690i 1680610198437487616
inverter_1,sensor_id=load_power value=380i 1680610198457868779
total,sensor_id=battery_power value=0i 1680610199692096470
total,sensor_id=battery_state_of_charge value=100i 1680610199693450692
inverter_1,sensor_id=pv_power value=4073i 1680610199715735404
inverter_1,sensor_id=grid_power value=-3690i 1680610199739595464
inverter_1,sensor_id=load_power value=380i 1680610199760904040
total,sensor_id=battery_power value=0i 1680610200995009843
total,sensor_id=battery_state_of_charge value=100i 1680610200999695075
inverter_1,sensor_id=pv_power value=4073i 1680610201047476972
inverter_1,sensor_id=grid_power value=-3690i 1680610201066022123
inverter_1,sensor_id=load_power value=380i 1680610201088875438
total,sensor_id=battery_power value=0i 1680610201597641573
total,sensor_id=battery_state_of_charge value=100i 1680610201599038776
inverter_1,sensor_id=pv_power value=4079i 1680610201623592197
inverter_1,sensor_id=grid_power value=-3690i 1680610201659956751
inverter_1,sensor_id=load_power value=380i 1680610201676648965
total,sensor_id=battery_power value=0i 1680610202932405728
total,sensor_id=battery_state_of_charge value=100i 1680610202934258517
inverter_1,sensor_id=pv_power value=4079i 1680610202967609724
inverter_1,sensor_id=grid_power value=-3690i 1680610203002737906
inverter_1,sensor_id=load_power value=380i 1680610203022110324
total,sensor_id=battery_power value=0i 1680610204217513196
total,sensor_id=battery_state_of_charge value=100i 1680610204220511321
inverter_1,sensor_id=pv_power value=4079i 1680610204243098602
inverter_1,sensor_id=grid_power value=-3700i 1680610204275526451
inverter_1,sensor_id=load_power value=370i 1680610204294031122
total,sensor_id=battery_power value=0i 1680610204811475396
total,sensor_id=battery_state_of_charge value=100i 1680610204814488391
inverter_1,sensor_id=pv_power value=4079i 1680610204838702724
inverter_1,sensor_id=grid_power value=-3700i 1680610204863320440
inverter_1,sensor_id=load_power value=370i 1680610204887906694
total,sensor_id=battery_power value=0i 1680610205500835558
total,sensor_id=battery_state_of_charge value=100i 1680610205502749069
inverter_1,sensor_id=pv_power value=4079i 1680610205526082451
inverter_1,sensor_id=grid_power value=-3700i 1680610205554524373
inverter_1,sensor_id=load_power value=370i 1680610205576804529
total,sensor_id=battery_power value=0i 1680610206786439740

Is there anywhere else I can look for errors?

15

One other thing, I’m not sure is relevant, when I do systemctl start or stop I am prompted for my password.
Is that normal?

16

Hmmm I don’t see anything in your config that would require a password. Are you asked by Telegraf? What happens if you specify your config explicitly to Telegraf e.g. telegraf --config my.conf --debug?