I am using telegraf with influxdb to write a json file through plugins. My problem is that when I start telegraf with my configuration an error pops up saying that it has failed to write the metrics and gives out an error:
I think it might be due to a problem with the json_time_format, although the error comes from the outputs.influxdb_v2 plugin. This is the telegraf.conf file:
[[inputs.file]]
files = [.../dummy2.log]
name_override = "TestJson"
data_format = "json"
json_query = "data"
tag_keys = [ "age:", "gender:", "mood:", "poseX", "poseY"]
json_name_key = "id:"
json_time_key = "date:"
json_time_format = "unix"
And this is an extract of the json file I am trying to write:
{
"sensor_name": "focus_1",
"date": "1615903935",
"data": [
{
"id:": 544,
"date:": "1615903935.0",
"age:": 76,
"gender:": 0,
"mood:": 2,
"poseX:": -0.51,
"poseY:": -0.22
},
{
"id:": 544,
"date:": 1615903936,
"age:": 76,
"gender:": 0,
"mood:": 0,
"poseX:": -0.85,
"poseY:": 0.98
}
}
I tried googling the error but nobody seems to be having anything similar. I am running everything in Ubuntu with Oracle VM Virtualbox. Thank you very much.
The first thing I would do is fix the timestamp problems.
It looks like some timestamps date: have a float value?
It could be that the parser has a problem with this.
Thank you for your reply. Timestamp does seem to be the problem. When I remove it and comment the time parser options in the telegraf.config it seems to be working, but I do need those time keys.
I tried changing them to a string, an int and a double but telegraf will not have any of it. I use these 2 data elements so I can at least compare results. When I do a test, it does look like it recognises everything but for some reason it still fails at writing the metrics and giving out that odd 422 error.
I still haven’t been able to write anything to influx via telegraf and json that contains any timestamp, no matter which format the timestamp is in (unix or other), the error is the same, Unprocessable entity.
I would proceed systematically. If something doesn’t work, don’t increase the complexity, but reduce it.
- forget your json, first check whether the connection to InfluxDB works at all. Take a simple standard input plugin (cpu) and test with it.
- add the file input plugin and test with a simple json snippet.
- add your special json and adjust the parser accordingly.
Can you change the content of the json files?
I don’t think this will work if the datatype of date is sometimes an integer and sometimes a float as a string. I suspect the parser won’t be able to cope with that.
The old saying: garbage in - garbage out 
I would also remove all the colons within the json key names - even though this should not be a problem in theory.
If the output of the json files cannot be changed, it might be possible to read in the json file lines as a string value and then parse it yourself in a custom processors.starlark or processors.execd plugin.
[[inputs.file]]
files = ["dummy2.log"]
name_override = "TestJson"
data_format = "value"
data_type = "string"
Thank you for your advice. I followed it and made sure that the connection to InfluxDB works. I tested the standard [[inputs.cpu]] plugin and it shows up in the Data Explorer. I wrote my current parser the way you mention, from little to big. I started with a simple parser and json using the documentation examples and built on that, but the moment I put a time key in the input plugin the data stops being written.
I can change the json so I tried removing the colons and sticking with integers for the unix time key. I’m just not sure what I’m doing wrong. Does someone have a working example of a simple json with a time key in the right format and a telegraf.conf file that can process it?
I’m asking this because I also tried using these documentation examples:
And I get that output when I use the -test flag, but I still get the error 422, so I’m really running out of ideas here -.-’
Do you get the error 422 also with the example data (not your json data) ?
If yes, please post the full error message here in markdown format:
```log
put the log lines here
```
And don’t post pictures of your logs or configs…
Actually, what I get from the example data is this:
2021-03-17T14:33:42Z I! Starting Telegraf 1.17.3
2021-03-17T14:33:42Z D! [agent] Initializing plugins
2021-03-17T14:33:42Z D! [agent] Starting service inputs
2021-03-17T14:33:42Z E! [inputs.file] Error in plugin: invalid character '}' looking for beginning of object key string
2021-03-17T14:33:42Z D! [agent] Stopping service inputs
2021-03-17T14:33:42Z D! [agent] Input channel closed
2021-03-17T14:33:42Z D! [agent] Stopped Successfully
2021-03-17T14:33:42Z E! [telegraf] Error running agent: input plugins recorded 1 errors
So I change the example json in a way it doesn’t get that error. If I change the json so the input plugin can process it what I get is this:
2021-03-17T14:17:22Z I! Starting Telegraf 1.17.3
2021-03-17T14:17:22Z I! Loaded inputs: file
2021-03-17T14:17:22Z I! Loaded aggregators:
2021-03-17T14:17:22Z I! Loaded processors:
2021-03-17T14:17:22Z I! Loaded outputs: influxdb_v2
2021-03-17T14:17:22Z I! Tags enabled: host=ruben-VirtualBox
2021-03-17T14:17:22Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"ruben-VirtualBox", Flush Interval:10s
2021-03-17T14:17:22Z D! [agent] Initializing plugins
2021-03-17T14:17:22Z D! [agent] Connecting outputs
2021-03-17T14:17:22Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2021-03-17T14:17:22Z D! [agent] Successfully connected to outputs.influxdb_v2
2021-03-17T14:17:22Z D! [agent] Starting service inputs
2021-03-17T14:17:32Z E! [outputs.influxdb_v2] Failed to write metric (will be dropped: 422 Unprocessable Entity): unprocessable entity: failure writing points to database: partial write: points beyond retention policy dropped=2
2021-03-17T14:17:32Z D! [outputs.influxdb_v2] Wrote batch of 2 metrics in 14.74234ms
2021-03-17T14:17:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
^C2021-03-17T14:17:37Z D! [agent] Stopping service inputs
2021-03-17T14:17:37Z D! [agent] Input channel closed
2021-03-17T14:17:37Z I! [agent] Hang on, flushing any cached metrics before shutdown
2021-03-17T14:17:37Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2021-03-17T14:17:37Z D! [agent] Stopped Successfully
There are no other error messages or logs that I’m aware of. I get all the feedback from the telegraf command with the -debug flag.
- I have changed the example data json to the following and now it seems to be working, it writes them both and I can see them in the Data Explorer
:
[
{
"a": 5,
"b_time":"17 Mar 21 15:04 MST"
},
{
"a": 7,
"b_time":"17 Mar 21 18:04 MST"
}
]
But if I change any of them to 16 of March for example, it gives me the 422 error.
- I haven’t touched anything regarding retention policies (I’m sure I wouldn’t even know where to start) so I’m guessing I’m using the default one that came when creating the bucket in the InfluxDB UI.
Does this mean that I cannot write data that is one day old? Is that because of the retention policies?
Yes, maybe.
If the timestamp is older than the retention policy, it is not even written to the database. What for?
Please check the InfluxDB to see which retention policy is set for this bucket.
I tried your json data and your config myself and it works for me, both in InfluxDB and in local written file, see details below and check against it if necessary.
json file:
{
"sensor_name": "focus_1",
"date": "1615903935",
"data": [
{
"id:": 544,
"date:": "1615903935.0",
"age:": 76,
"gender:": 0,
"mood:": 2,
"poseX:": -0.51,
"poseY:": -0.22
},
{
"id:": 544,
"date:": 1615903936,
"age:": 76,
"gender:": 0,
"mood:": 0,
"poseX:": -0.85,
"poseY:": 0.98
}
]
}
telegraf.conf:
[agent]
interval = "1s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "1s"
flush_jitter = "1s"
precision = "ns"
debug = true
logtarget = "file"
logfile = "telegraf.log"
hostname = "localhost"
omit_hostname = true
[[inputs.file]]
files = ["json.json"]
name_override = "TestJson"
data_format = "json"
json_query = "data"
tag_keys = [ "age:", "gender:", "mood:", "poseX", "poseY"]
json_name_key = "id:"
json_time_key = "date:"
json_time_format = "unix"
[[outputs.file]] # only for debugging
files = ["json.out"]
influx_sort_fields = true
[[outputs.influxdb_v2]]
urls = "removed"
token = "removed"
organization = "removed"
bucket = "removed"
json.out:
TestJson,age:=76,gender:=0,mood:=2 id:=544,poseX:=-0.51,poseY:=-0.22 1615903935000000000
TestJson,age:=76,gender:=0,mood:=0 id:=544,poseX:=-0.85,poseY:=0.98 1615903936000000000
Wow, IT WORKS! I can’t believe it, it was the retention policy thing, it was set to 24h so I changed it to a year and now it writes all my data xD Thank you so much!! This was driving me crazy, I couldn’t find anything related. Thank you thank you so very much!
I could have figured it out earlier from the error message, but the inconsistent timestamp led me down the wrong path at first.
Therefore, always test step by step first and only increase the complexity when the simple things work. 
