Help with decoding JSON mqtt data

Telegraf
, ,
1

Hello,
I am getting hexadecimal 32bit float types via mqtt in json format.

The json format looks like this:

[ {
“bn” : “urn:dev:DEVEUI:000000C00001B08A7:”,
“bt” : 1.614612404E9
}, {
“n” : “payload”,
“vs” : “0100100100”
}, {
“n” : “port”,
“v” : 2.0
} ]

How can i get “vs” : “0100100100” with the value being the hexadecimal code minus the first two bytes.

So the first two numbers need to be stripped, and the resulting 32 bit hexademical value needs to be converted into a 32bit float dcab value.

My telegraf plugin is a simple [[inputs.mqtt_consumer]] with
data_format = “json”
json_string_fields = [“vs”]

I can see the raw data value finally, but i have no idea how to convert it into an actual number.

2
  • Are you sure you mean bytes and not possibly bits? Looks more binary to me?
  • So the json data ends up correctly in the database? Or is there already a problem? Where do you “see” the raw data?
  • Which version of InfluxDB?
  • Where do you want to convert the raw data? Already in Telegraf? Or in InfluxDB?
3

Hello, Thanks for your repy!
I can see the raw data like this:
inf

data influxdbcloud
data influxdbcloud3344×679 38.4 KB

It are bytes given in hex format. the first two characters are the device ID.

the other 8 are hex bytes. I would like to conver them to a float. The data is encoded in float32 dcba format.

4

I am using cloud 2.0 for testing right now. The values are 00 00 00 00 because the test was zero. an acutally value would be: 01 81 26 1a 42. Which would be a string like this: 0181261a42. The mqtt JSON message would be:
[{"bn":"urn:dev:DEVEUI:0059AC54001B07A6:","bt":1.624629038E9},{"n":"payload","vs":"01864e2440"},{"n":"port","v":2.0}]

Where i would only need: vs":"01864e2440 as data.

5

The string should be stripped of the first two characters, and the resulting 864e2440 should be converted to a float like this: Float - Little Endian (DCBA). which would result in: 2.56729269

6

Ok, my first reaction was, this should be easily doable in flux.

Unfortunately it turns out to be very difficult. :astonished:
The easy part would be to split the string with strings.substring() to cut out the desired substring.
But then it gets difficult…
It turns out, that flux does not offer any operators/converters for binary or hexadecimal data types… :expressionless:
After some searching in the flux Reference, I tend to come to the conclusion that I would do this right at the time of capture in Telegraf with a custom plugin.
For this would then come into question:

  • A custom inputs.execd Plugin
  • A custom processors.execd Plugin
  • A custom processors.starlark Plugin

I would try Starlark first.
Unless someone has a better idea… :wink:

EDIT:
I think after a quick look at the Starlark Spec, that there will be the same problem there… :disappointed_relieved:

7

In Python 3, this would be a simple matter :wink:

from struct import unpack
vs = "01864e2440"
stripped = vs[2:]
binary_data = bytes.fromhex(stripped)
tofloat = unpack('<f', binary_data)[0]
print(tofloat)

If you know Python, you could put this snippet into a custom inputs or processors plugin.

8

Hello,
I know this is an older thread, but this was the closest to my use case. So for anyone in IoT with this challenge, here is my starlark solution for decoding to IEEE float from a hex string.

I use it within a .starlark script, and within telegraf initiate it by processors.starlark.


def converter(text):

        d4 = int(text[0:2], base=16)
        d3 = int(text[2:4], base=16)
        d2 = int(text[4:6], base=16)
        d1 = int(text[6:], base=16)

        if (d1 & 128)==128:
                sign=-1
        else:
                sign=1

        exp=(d1<<1)+(d2>>7)
#       mantissa=(((d2 & 127)*(2**16) + d3*(2**8) + d4))/(2**23)
        mantissa=(((d2 & 127)*65536 + d3*256 + d4))/8388608

        result=sign*(1+mantissa)*math.pow(2,(exp-127))
        return result