Adding a unique tag doesn't change the export behaviour

Telegraf
1

Hello,
I try to make Telegraf export multiple counters, which arrive at the same time interval and don’t have unique tags. By default, this makes Telegraf to overwrite all counters with the last one.

This is from a raw capture, e.g. “infront of” Telegraf. It show the number of unique counters in a batch. The file is also attached here.

$ grep counter-name -A 4 np_stats_raw.txt | sed "/}\|{\|name\|--/d" | sort | uniq | wc -l 76

The unmodified understanding of Telegraf for tags/fields. This combination of tags is not unique across the different counters in the same batch.

{"fields":{"nodes/node/nps/np/counters/np_counter/counter_index":2046,"nodes/node/nps/np/counters/np_counter/counter_name":"UNKNOWN_L2_ON_L3_DISCARD","nodes/node/nps/np/counters/np_counter/counter_type":"FWD","nodes/node/nps/np/counters/np_counter/counter_value":108,"nodes/node/nps/np/counters/np_counter/rate":0},"name":"np_stats","tags":{“host”:“prod-lab-jump02”,“node_name”:“0/0/CPU0”,“np_name”:“np0”,“path”:“Cisco-IOS-XR-asr9k-np-oper:/hardware-module-np/nodes/node/nps/np/counters”,“source”:“asr-1”},"timestamp":1589291734}

Adopting a unique field as a tag: This, unfortunately, doesn’t change anything, the export includes only the last counter’s data.

{"fields":{"nodes/node/nps/np/counters/np_counter/counter_index":2046,"nodes/node/nps/np/counters/np_counter/counter_type":"FWD","nodes/node/nps/np/counters/np_counter/counter_value":108,"nodes/node/nps/np/counters/np_counter/rate":0},"name":"np_stats","tags":{“host”:“prod-lab-jump02”,“node_name”:“0/0/CPU0”,“nodes/node/nps/np/counters/np_counter/counter_name”:“UNKNOWN_L2_ON_L3_DISCARD”,“np_name”:“np0”,“path”:“Cisco-IOS-XR-asr9k-np-oper:/hardware-module-np/nodes/node/nps/np/counters”,“source”:“asr-1”},"timestamp":1589348462}

Can you help me understand, what I miss here, please?

np_stats_raw.txt.gz (42.5 KB)

2

Hello @bofh,
I’m having trouble understanding your problem. Can you please help me, by taking a step back and describing the general problem?
Thank you.

3

A serie is defined by unique set of tags and a timestamp, correct?

So, if you have multiple data (set of fields), arrived at the same (timestamp), they need to be accompanied by such set. Otherwise, only the last one will be exported to the database.
For example - if this arrives at time T0 and tags t0, t1 and t2 are not unique. Fields f0 and f1 are unique and have different values in T0.

T0; t0, t1, t2; f0, f1, f2 - non-unique tags values/unique fields values
…
T0; t0, t1, t2; f0, f1, f2 - non-unique tags values/unique fields values

This results in only the last raw to be exported to Influx.

If we redefine one field as tag (f0 -> t3), then we have a unique set of tags for a unique set of fields in T0.

T0; t0, t1, t2, t3; f1, f2 - unique tags values/unique fields values
…
T0; t0, t1, t2, t3; f1, f2 - unique tags values/unique fields values

By having this, Telegraf should be able to distinct the different fields data and export the whole set it receives at T0, by not overriding a single entry.

This is not the case, unfortunately. In line pasted JSON output form Telegraf shows these two scenarios (non-unique/unique tag set). The result is the same - only the last raw of T0 data is exported to Influx.

Attached is a raw data capture in JSON format, in front of Telegraf.

HTH