Transform <a field>=<a value> into field=<a field> value=<a value>

Telegraf
1

Hi,

this looks quite simple, but I couldn’t find the right plugin.

I have this line:

Converter,host=23d86d9833d1,type=TF,zone=01 Frequency=401i,Quality="OK (0x0)" 1695986031000000000

and I want to convert it to:

Converter,host=23d86d9833d1,type=TF,zone=01 measurement="Frequency",value=401i,Quality="OK (0x0)" 1695986031000000000

So that I get two columns in my database instead of one.

How do I do that? Thanks!

2

This is what the unpivot processor does:

[[processors.unpivot]]
  tag_key = "measurement"
  value_key = "value"

Transforms

Converter,host=23d86d9833d1,type=TF,zone=01 Frequency=401i,Quality="OK (0x0)" 1695986031000000000

to

Converter,host=23d86d9833d1,measurement=Frequency,type=TF,zone=01 value=401i 1695986031000000000
Converter,host=23d86d9833d1,measurement=Quality,type=TF,zone=01 value="OK (0x0)" 1695986031000000000

Of course, that did that to both fields, Frequency and Quality. Also note the measurement value, because it is a string, becomes a tag. You can covert it back to a field if you need it to be a field instead.

If you know the names of the fields you want to do this with, you could also do it with the starlark processor.

3

Thanks for the answer, but that is not what I need.

First, we can ignore the quality value. This comes from an OPC UA input and could be filtered out.
The target output is TimescaleDB (via Postgres)

This line:

Converter,host=23d86d9833d1,measurement=Frequency,type=TF,zone=01 value=401i

would give me a tag ‘measurement’, but what I need is a field ‘measurement’ (both with the value ‘Frequency’).

The additional tag is a problem because the Timescale output creates a unique tag_id from it. But I have other values (like ‘Current’) that need to have the same ID. So I can’t have that additional tag.

4

As I mentioned above, you can then convert it back.

5

Oh, I see. After ‘unpivot’ I run a ‘converter’ processor to convert the ‘measurement’ tag to a field of the same name. Ok, thanks.

Sorry, one last question: is it more efficient to run two processors like this or one starlark script that does the same?

Thanks!

6

Exactly! I would also mention setting the “order” value in your config to ensure it runs afterwards, something like:

[[processors.unpivot]]
  order = 1
  tag_key = "measurement"
  value_key = "value"

[[processors.converter]]
  order = 2
  ...

Sorry, one last question: is it more efficient to run two processors like this or one starlark script that does the same?

I would generally say using these processors is preferred over running starlark. Starlark has a bit of overhead.

1 Like