[solved] Telegraf -> Influx write issue

Store
1

Hi, i am new here and i dont know if i am right here. I am using Telegraf to get data via http (json data) and writing these into influx db with windows. If i test my config like below all is good.

C:“Program Files”\Telegraf\telegraf.exe --config C:“Program Files”\Telegraf\telegraf.conf --test

Received example data entry:

data_traffic,url=myurl lastvalue=“323 GB”,lastvalue_raw=323.1695,objid=25264,probe=“Testship” 1557299254000000000

But if i check the measurement only one entry is inside. I dont know why! I hope somebody can help me. Thanks in advance…

2

My Telegraf Config file:

Blockquote

Global tags can be specified here in key=“value” format.

[global_tags]
#dc = “test”

Configuration for telegraf agent

[agent]
interval = “60s”
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = “0s”
flush_interval = “10s”
flush_jitter = “5s”
precision = “”
debug = true
quiet = false
logfile = “/Program Files/Telegraf/telegraf.log”
hostname = “”
omit_hostname = true

###############################################################################

OUTPUTS

###############################################################################

Configuration for sending metrics to InfluxDB

[[outputs.influxdb]]
urls = [“http://127.0.0.1:8086”]
database = “database_test”

skip_database_creation = false

retention_policy = “”
timeout = “2s”

write_consistency = “any”

###############################################################################

INPUTS

###############################################################################

[[inputs.http]]

One or more URLs from which to read formatted metrics

urls = [
“myurl”
]

HTTP method

method = “GET”

timeout = “1m”

data_format = “json”

name_override = “data_traffic”

json_string_fields = [“probe”, “lastvalue”]

json_query = “sensors”

json_timezone = “Local”

Blockquote

3

hi @joschika77 ,welcome ,
do you have timestamps in your json files ?
Have you checked the measurements again after 60 seconds ?
Do you see errors in /Program Files/Telegraf/telegraf.log ?

best regards

4

This is an excerpt from the json file:

Blockquote

{
“some-version”: “18.4.47.1962”,
“treesize”: 51,
“sensors”: [
{
“objid”: 25264,
“probe”: “some text”,
“group”: “some text”,
“name”: “some text”,
“lastvalue”: “245 GB”,
“lastvalue_raw”: “244.94”
},
{
“objid”: 40684,
“probe”: “some text”,
“group”: “some text”,
“name”: “some text”,
“lastvalue”: “199 GB”,
“lastvalue_raw”: 198.9999
}
}

Blockquote

5

Hi,

i’ve checked the measurement (is only one). After the 60s he insert the same entry again. Every time only the first item from the json list. I dont have errors in my logfile.

regards

6

This is an excerpt from the logfile:

Blockquote

2019-05-08T09:18:45Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:“”, Flush Interval:10s
2019-05-08T09:18:45Z D! [agent] Connecting outputs
2019-05-08T09:18:45Z D! [agent] Attempting connection to output: influxdb
2019-05-08T09:18:45Z D! [agent] Successfully connected to output: influxdb
2019-05-08T09:18:45Z D! [agent] Starting service inputs
2019-05-08T09:19:00Z D! [outputs.influxdb] wrote batch of 51 metrics in 57.8857ms
2019-05-08T09:19:00Z D! [outputs.influxdb] buffer fullness: 0 / 10000 metrics.

Blockquote

7

the logfile looks ok ,
what is the version of telegraf you are using ?

8

I think its Telegraf 1.10.3. It is maybe a Windows version issue?

9

Hi ,

it is because there are no unique tags defined in the telegraf config …
your tag is url=myurl ,
the timestamp is also the same at the moment the entries are inserted .
This causes some entries to be overwritten ( updated )
and only one record will be written to the database .

I think you can solve it by using

 tag_keys = ["objid"]

please also note that in your example you have an entry with less double quotes …

“lastvalue_raw”: “244.94”
“lastvalue_raw”: 198.9999

that will cause data type conflictions in your measurement
and data will be rejected

10

Yeah thanks a lot it works fine now. The “tag_keys” is the index. Understood.

11

thank you for your feedback , glad it works :slight_smile: it is true , tag keys are indexed