MQTT Parse with Timestamp not working (Telegraf Noob)

1

Hi,

i try to configure my mqtt consumer with a timestamp. Other mqtt consumers already work.

In Node-Red i get date and State input on the Dashboard. After it will converted to json. This is the mqtt publish:

Topic: manually/zahlstand

Value: {“date”:1643590800000,“state”:8913}

The Timestamp is unix-ms and only the date: dd/mm/yyyy

How i can parse it? I have no idea. This is my Config:


[[inputs.mqtt_consumer]]
client_id = "telegraf2"
servers = ["tcp://IP:1883"]
#qos = 0
name_override = "mqtt_zaehlerstand"
topics = ["manually/#",]
data_format = "json_v2"
username = "user"
password = "verystrongpassword"
[[inputs.mqtt_consumer.topic_parsing]]
topic = "+/+"
tags = "state/date"
[[inputs.mqtt_consumer.json_v2]]
timestamp_path = "date"
timestamp_format = "unix_ms"
[[inputs.mqtt_consumer.json_v2.field]]
path = "position"
rename = "p"
type = "int"

I tried something but nothing works.

If my config is not good generally you can give me better version, if you want.

Thanks

2

Hi, I have a similar issue: I’m using MQTT and Node-RED and also am having difficulty with timestamp. Are you using node-red-contrib-influxdb (which I am) or are you using http and the API? With my Influxdb1.8 instance, I can add the timestamp as a key value but it has to be the last element in the payload object. Your timestamp above works out to Monday, 31 January 2022 01:00:00, which I think should be valid (or it would be for my 1.8 instance). Try swapping around your elements so they become

Value: {“state”:8913, “time”:1643590800000}

I’m guessing you might have to change around your tags as well although, if you are using the timestamp, then date is redundant as a tag. Also, where do you specify the measurement for this datapoint? In my MQTT message, I can set msg.measurement and the 1.8 instance handles the message and payload as I would expect.

My issue is that I’m migrating to Influxdb 2.3 OSS and some of my test data was visible in my bucket but as I’ve been trying to get timestamp to work, I’ve been deleting the bucket from the web browser interface and recreating it to see what affect the Node-RED changes have made. Now the messages are being sent from the Node-RED Influxdb Out node (ssame as before) but not appearing in the bucket any more.

3

Hi,

no i am using simple dashboard, join and change modules. Dashboard Date and Numberinput, Change Topicnames, Join, cause the Inputs need to come in one Time. The Join change the Input to: {“date”:1643590800000,“state”:8913} and send it to mqtt.

The Timestamp Date is unix-ms i think. But i have no idea to parse it in Telegraf for InfluxDB and Grafana. Without the Timestamp the Data will not be on the right Place.

grafik
grafik848×176 12.9 KB 

[
    {
        "id": "a3601b868b3443f7",
        "type": "comment",
        "z": "e954db9cd434d6f3",
        "name": "Zählerstand",
        "info": "",
        "x": 310,
        "y": 1680,
        "wires": []
    },
    {
        "id": "96841b65b8917f41",
        "type": "ui_date_picker",
        "z": "e954db9cd434d6f3",
        "name": "",
        "label": "Datum des Zählerstandes",
        "group": "5934e2670cdcc8dc",
        "order": 8,
        "width": 0,
        "height": 0,
        "passthru": true,
        "topic": "date",
        "topicType": "msg",
        "className": "",
        "x": 350,
        "y": 1740,
        "wires": [
            [
                "339f4f1f01f6db42"
            ]
        ]
    },
    {
        "id": "74c6308e0d42c692",
        "type": "join",
        "z": "e954db9cd434d6f3",
        "name": "",
        "mode": "custom",
        "build": "object",
        "property": "payload",
        "propertyType": "msg",
        "key": "topic",
        "joiner": "\\n",
        "joinerType": "str",
        "accumulate": true,
        "timeout": "",
        "count": "2",
        "reduceRight": false,
        "reduceExp": "",
        "reduceInit": "",
        "reduceInitType": "",
        "reduceFixup": "",
        "x": 710,
        "y": 1760,
        "wires": [
            [
                "fc60cb31e8c418a5",
                "5f6eb6e898ee9859"
            ]
        ]
    },
    {
        "id": "daeca697d0b28a24",
        "type": "ui_text_input",
        "z": "e954db9cd434d6f3",
        "name": "",
        "label": "Zählerstand",
        "tooltip": "",
        "group": "5934e2670cdcc8dc",
        "order": 8,
        "width": 0,
        "height": 0,
        "passthru": true,
        "mode": "number",
        "delay": 300,
        "topic": "state",
        "sendOnBlur": true,
        "className": "",
        "topicType": "msg",
        "x": 310,
        "y": 1780,
        "wires": [
            [
                "26b3f992110a2ef1"
            ]
        ]
    },
    {
        "id": "339f4f1f01f6db42",
        "type": "change",
        "z": "e954db9cd434d6f3",
        "name": "date",
        "rules": [
            {
                "t": "set",
                "p": "topic",
                "pt": "msg",
                "to": "date",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 550,
        "y": 1740,
        "wires": [
            [
                "74c6308e0d42c692"
            ]
        ]
    },
    {
        "id": "26b3f992110a2ef1",
        "type": "change",
        "z": "e954db9cd434d6f3",
        "name": "state",
        "rules": [
            {
                "t": "set",
                "p": "topic",
                "pt": "msg",
                "to": "state",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 550,
        "y": 1780,
        "wires": [
            [
                "74c6308e0d42c692"
            ]
        ]
    },
    {
        "id": "5f6eb6e898ee9859",
        "type": "mqtt out",
        "z": "e954db9cd434d6f3",
        "name": "",
        "topic": "manually/zahlstand",
        "qos": "",
        "retain": "",
        "respTopic": "",
        "contentType": "",
        "userProps": "",
        "correl": "",
        "expiry": "",
        "broker": "8aa7c4c7.b6d308",
        "x": 930,
        "y": 1760,
        "wires": []
    },
    {
        "id": "fc60cb31e8c418a5",
        "type": "debug",
        "z": "e954db9cd434d6f3",
        "name": "Debug Join",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "",
        "statusType": "auto",
        "x": 910,
        "y": 1720,
        "wires": []
    },
    {
        "id": "5934e2670cdcc8dc",
        "type": "ui_group",
        "name": "Stromzähler",
        "tab": "7c6fa6e7.e5b808",
        "order": 7,
        "disp": true,
        "width": "6",
        "collapse": false,
        "className": ""
    },
    {
        "id": "8aa7c4c7.b6d308",
        "type": "mqtt-broker",
        "name": "Mosquitto",
        "broker": "172.50.0.5",
        "port": "1883",
        "clientid": "",
        "autoConnect": true,
        "usetls": false,
        "compatmode": false,
        "protocolVersion": "4",
        "keepalive": "60",
        "cleansession": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "birthMsg": {},
        "closeTopic": "",
        "closeQos": "0",
        "closePayload": "",
        "closeMsg": {},
        "willTopic": "",
        "willQos": "0",
        "willPayload": "",
        "willMsg": {},
        "sessionExpiry": ""
    },
    {
        "id": "7c6fa6e7.e5b808",
        "type": "ui_tab",
        "name": "Home",
        "icon": "dashboard",
        "order": 1,
        "disabled": false,
        "hidden": false
    }
]
4

I haven’t dabbled with Telegraf so far but you could use the dashboard Join etc and the pass the message on to the node-red-contrib-influxdb node to go to Influxdb

Screenshot 2022-08-06 174306
Screenshot 2022-08-06 174306723×124 17.6 KB

This is the way I get my data into Influxdb 1.8 but I am struggling getting it into 2.3 for some reason. The data I’m sending to Influxdb 1.8 has the time field as the last element of the object and it does get handled as a timestamp.

5

This is not usable for me. I need to convert my date input to timestamp in the table. The Influxdb doesnt do it alone.

6

The graphic was intended to illustrate that to get the data from the table to Influxdb using a node, you don’t have to use telegraf.

If you need to manipulate a date/time, then using the Date/Time Formatter from node-red-contrib-moment provides excellent ways to manipulate a simple text date and turn it into a properly formatted timestamp. It also deals very well with timezones if you need to work with UTC for the timestamp.

7

I no need to manipulate Time. I need to input a State with timestamp and the Timestamp needs to be my input Date.
I already tested “Moment” from Node-Red and this not do, what i need.

At the Moment i have 6 Data Points.
For Example:

04.02.2014 - 4000
02.06.2016 - 5000
01.01.2019 - 8000
31.01.2022 - 13000

If i input this with moment, i only get a value with the date. Grafana doesnt care this and the graph is not correct.

8

OK, just build your package with the state key value first and the time key value at the end and Influxdb should recognise the timestamp and use it as such.

9

This dont help me to parse my mqtt telegraf config correct.