Hello everyone, i have quite troubled for the past days regarding the problem above.
the code above supposedly make input with the specific 3rd part of topic which is “kedalaman”, “meter1”, “meter2”, or “meter3” appeared in influxdb cloudless. but went i tested by input values on that specified topic, only “kedalaman” topic that going in influx cloudless not the other 3. because i new to starlark code idk how to make it so other specified topic can be appeared in influx as well.
this is the metrics,
i does several test with another topic other than that one and apparently it didnt send any errors or some kind that tell me it didnt succesfully sent
here the history of me sending some input with different topics other than sensor/lokasi/kedalaman in 20:05 WIB to 20:08 Western Indonesian time
First use the regex processor to extract out the sensor tag. This assumes that your topics are of the format /sensor/host/name. It will create a temporary tag called sensor with the value name.
Then using starlark take the sensor tag value and replace the value field. So the field value=42 and tag sensor=meter2 becomes the field meter2=42.
[[processors.regex]]
order = 1
[[processors.regex.tags]]
key = "topic"
pattern = '.*\/(.*)'
replacement = "${1}"
result_key = "sensor"
[[processors.starlark]]
order = 2
source = '''
def apply(metric):
# Replace "value" field with sensor name
if metric.tags.get("sensor") and metric.fields.get("value"):
metric.fields[metric.tags["sensor"]] = metric.fields["value"]
metric.tags.pop("sensor")
metric.fields.pop("value")
return metric
'''
Let me know if that works for you. This transforms:
I will add, that doing this is not ideal for InfluxDB. You are creating what we call “sparse” metrics. Meaning that your metrics have a lot of different field names that are not always present. This means that your metrics have lots of null fields if you think about it like a spreadsheet or table:
i try the code but somehow it give me this error in that above code
error parsing data: line 58: invalid TOML syntax
and also is it fine to add [processors.starlark] below the [[processors.regex.tags]]
we can see in this quote that the color between processors.starlark and processor.regex.tags is the same but went i try it on telegraf that in influxdb cloud the color of processor.starlark is the same as the rest as the line code. is it fine like that?
i didnt quite get it on the commented code about changing “value” with sensor name am i needed to change that into sensor name or just let the code stay at it is
Well went i test it with different topics (after i change the code like you suggested) like sensor/lokasi/depth, sensor/lokasi/length, sensor/lokasi/meter
It didn’t register to influx it like it ignored the different topics completely beside sensor/lokasi/kedalaman.
I already test it in different bucket the problem still persist
I use output.file and i didnt see other topics other than that specific topic so I’m not sure how to tell what’s wrong with it because there no error or indication that the message with other topics failed to sent.
I saw and input several topics getting delivered only on mqtt consumer but only that specific topic get processed by telegraf
after the changed i found that only “sensor/lokasi/kedalaman” and “sensor/lokasi/kedalaman/#” is going in my table. and with “sensor/lokasi/kedalaman/#” any topic after kedalaman work like in example
so i’m not gonna change that anytime soon
so since this code will have a lot of null values in the fields that i worked with is there a way to not show that null values went try to present that fields in influx cloud or any other visual tool? (sorry if the above question is confusing beforehand)
The correct way to handle this is to use the originally structured value field and the topic tag. This way you can filter on a topic tag and the value field will show you your values.
